Internal validation parameters of linear regression equations in QSAR problem
Abstract
The article discusses a set of internal validation parameters that are (or can be) used to describe the quality of regression models in quantitative structure-activity relationship problems. Among these parameters there are well known determination coefficient, root mean square deviation, mean absolute error, etc. Also the indices based at Kullback-Leibler divergence as a measure of distance between two sets have been investigated. All the parameters (indices) were calculated for several regression models which describe boiling point of saturated hydrocarbons (alkanes). Regression models include a four-component additive scheme and equations describing the property as a function of topological indices. The two types of regressions based on these indices are linear dependencies on only one topological index and linear dependencies on topological index and the number of carbon atoms in the hydrocarbon. Various linear regression equations have been described with internal validation parameters that evaluate the quality of the equations from different perspectives. It is shown that a wide set of test parameters is not only an additional yet alternative description of regression models, but also provides the most complete description of the predictive characteristics and quality of the obtained regression model.
Downloads
References
Tropsha A., Gramatica P. and Gombar V. The Importance of Being Earnest: Validation is the Absolute Essential for Successful Application and Interpretation of QSPR Models, QSAR Comb. Sci. 2003, 22, 69-77. https://doi.org/10.1002/qsar.200390007
Golbraikh A., Tropsha A. Beware of Q2! Journal of Molecular Graphics and Modelling. 2002, 20, 269–276. https://doi.org/10.1016/S1093-3263(01)00123-1
Alexander D. L. J., Tropsha A., and Winkler D. A. Beware of R2: Simple, Unambiguous Assessment of the Prediction Accuracy of QSAR and QSPR Models. J. of Chem. Inform. and Model. 2015, 55(7), 1316-1322. https://doi.org/10.1021/acs.jcim.5b00206
Joseph V. R. and Vakayil A. SPlit: An Optimal Method for Data Splitting. Technometrics. 2021, 64(2), 166-176. https://doi.org/10.1080/00401706.2021.1921037
Liu H., Cocea M. Semi-random partitioning of data into training and test sets in granular computing context. Granul. Comput. 2017, 2, 357–386. https://doi.org/10.1007/s41066-017-0049-2
Joseph V. R. Optimal ratio for data splitting. Stat. Anal. Data Min.: ASA Data Sci. J. 2022, 15, 531–538. https://doi.org/10.1002/sam.11583
Anscombe F. J. Graphs in Statistical Analysis. Am. Stat. 1973, 27, 17-21. https://doi.org/10.2307/2682899
Besalu E., Julian-Ortiz J. V., Pogliani L. Trends and Plot Methods in MLR Studies. J. Chem. Inf. Model. 2007, 47, 751-760. https://doi.org/10.1021/ci6004959
Mukwembi S., Nyabadza F. A new model for predicting boiling points of alkanes, Scientific Reports, 2021, 11, 24261. https://doi.org/10.1038/s41598-021-03541-z
Mukwembi S, Nyabadza F. Predicting anti‑cancer activity in flavonoids: a graph theoretic approach. Scientific Reports. 2023, 13, 3309. https://doi.org/10.1038/s41598-023-30517-y
Zhen W., Khalid A., Ali P., Rehman H., Siddiqui M. K., Ullah H. Topological Study of Some Covid-19 Drugs by Using Temperature Indices. Polycyclic Aromatic Compounds. 2022. 43 (2), 1133-1144. 10.1080/10406638.2022.2025864
Zhang Y., Khalid A., Siddiqui M. K., Rehman H., Ishtiaq H., and Cancan M. On Analysis of Temperature Based Topological Indices of Some Covid-19 Drugs. Polycyclic aromatic compounds. 2023, 43(4), 3810–3826. https://doi.org/10.1080/10406638.2022.2080238
Todeschini R., & Consonni V. (2000). Handbook of Molecular Descriptors. Weinheim: Wiley-VCH.
Devillers J., & Balaban A. T. (1999). Topological Indices and Related Descriptors in QSAR and QSPR. London: CRC Press
Roy K., Kar S., Das N.R. A Primer on QSAR/QSPR Modeling. Fundamental Concepts. Springer: 2015
M.R. Spiegel, John J. Schiller, R. A. Srinivasan Probability and Statistics. McGraw-Hill, New York, 2013, 424 p.
Besalú E., de Julián-Ortiz J. V., Iglesias M., Pogliani. L. An overlooked property of plot methods. Journal of Mathematical Chemistry. 2006, 39, 475-484. https://doi.org/10.1007/s10910-005-9035-z
Hyndman R. J., Koehler A. B. Another look at measures of forecast accuracy, International Journal of Forecasting. 2006, 22, 679– 688. https://doi.org/10.1016/j.ijforecast.2006.03.001
Hyndman R. J. Another Look at Forecast Accuracy Metrics for Intermittent Demand, Foresight. 2006, 4, 43-46. https://robjhyndman.com/papers/foresight.pdf
Hewamalage H., Ackermann K., Bergmeir C. Forecast evaluation for data scientists: common pitfalls and best practices. Data Min Knowl Disc. 2023, 37, 788–832. https://doi.org/10.1007/s10618-022-00894-5
Kullback S. Information theory in statistics, Glouchester Mass, 1978, 399 p.
Kullback S. Leibler R.A. On information and sufficiency. Ann. Math. Statist. 1951, 22(1), 79 –86. https://doi.org/10.1214/aoms/1177729694
Hummer G., Garde S., Garcia A. E., Pohorille A., Prat L. R. An information theory model of hydrophobic interactions. Proc. Natl. Acad. Sci. USA. 1996, 93, 8951-8955. https://doi.org/10.1073/pnas.93.17.895
Arlot S., Celisse A. A survey of cross-validation procedures for model selection. Statistics surveys, 2010, 4, 40-79. https://doi.org/10.48550/arXiv.0907.4728
Quan N. T. The Prediction Sum of Squares as a General Measure for Regression Diagnostics. J. Business & Economic Statistics. 1988, 6(4), 501-504. https://doi.org/10.1080/07350015.1988.10509698
Stone M. Cross-Validatory Choice and Assessment of Statistical Predictions. Journal of the Royal Statistical Society. 1974, Ser. B, 36, 111-133. http://www.jstor.org/stable/2984809
Cawley G. C., Talbot N. L. C. On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation. Journal of Machine Learning Research. 2010, 11, 2079-2107. http://jmlr.org/papers/v11/cawley10a.html
