Annotation of Allosteric Compounds to Enhance Bioactivity Modeling for Class A GPCRs.

Proteins often have both orthosteric and allosteric binding sites. Endogenous ligands, such as hormones and neurotransmitters, bind to the orthosteric site, while synthetic ligands may bind to orthosteric or allosteric sites, which has become a focal point in drug discovery. Usually, such allosteric modulators bind to a protein noncompetitively with its endogenous ligand or substrate. The growing interest in allosteric modulators has resulted in a substantial increase of these entities and their features such as binding data in chemical libraries and databases. Although this data surge fuels research focused on allosteric modulators, binding data is unfortunately not always clearly indicated as being allosteric or orthosteric. Therefore, allosteric binding data is difficult to retrieve from databases that contain a mixture of allosteric and orthosteric compounds. This decreases model performance when statistical methods, such as machine learning models, are applied. In previous work we generated an allosteric data subset of ChEMBL release 14. In the current study an improved text mining approach is used to retrieve the allosteric and orthosteric binding types from the literature in ChEMBL release 22. Moreover, convolutional deep neural networks were constructed to predict the binding types of compounds for class A G protein-coupled receptors (GPCRs). Temporal split validation showed the model predictiveness with Matthews correlation coefficient (MCC) = 0.54, sensitivity allosteric = 0.54, and sensitivity orthosteric = 0.94. Finally, this study shows that the inclusion of accurate binding types increases binding predictions by including them as descriptor (MCC = 0.27 improved to MCC = 0.34; validated for class A GPCRs, trained on all GPCRs). Although the focus of this study is mainly on class A GPCRs, binding types for all protein classes in ChEMBL were obtained and explored. The data set is included as a supplement to this study, allowing the reader to select the compounds and binding types of interest.

Introduction

Drugs bind to proteins to exert a biological effect, which can be activation, deactivation, or blockage to prevent an endogenous ligand from binding. The location on the protein where the endogenous, or natural, ligand binds is called the orthosteric binding site. All other binding sites are described as allosteric binding sites.[1] Allosteric binding of ligands (allosteric modulation) has been reported and studied at many protein classes, including membrane-bound G protein-coupled receptors (GPCRs).[1−3] Moreover, structural information is available from the Protein DataBank,[4] which shows the binding of allosteric ligands to GPCRs at different sites (Figure ).[2,5] The transmembrane (TM) domains of the orthosteric binding pocket in GPCRs are highly conserved for most GPCR classes. An exception is formed by class C GPCRs, for which the orthosteric site lies in the extracellular Venus flytrap domain. The most common allosteric binding site for class C GPCRs is located close to or at the orthosteric site in the TM domain of other (class A) GPCRs.[6] Other GPCR allosteric binding sites have been identified close to or in the intracellular domain and between helices in the TM domain.[2,3] The physicochemical properties of these multiple binding sites can vary greatly, enabling dissimilar ligands to bind the same protein via different sites.

Figure 1

Ligand binding sites observed in crystal structures of class A GPCRs. Orthosteric ligands are shown in green, allosteric modulators in orange. The following crystal structures are included (PDB): 4MBS,[10] 4MQT,[10] 4N6H,[11] 4NTJ,[12] 4PHU,[13] 5LWE,[14] 5NDZ,[15] 5NLX,[16] 5T1A,[2] 5TZR,[17] 5TZY,[17] 5X7D,[18] 6C1Q,[19] and 6C1R.[20]

Although allosteric modulators are widely studied and reported, allosterism of compounds is not yet annotated in the public chemical database ChEMBL.[7] Previous attempts have been made to retrieve allosteric properties of compounds from the original published article to fill this information gap in the ChEMBL database.[8,9] However, although allosteric and orthosteric annotations were retrieved, the added value of this information in binding predictions was not assessed. We observed that allosteric compounds bind with a lower absolute affinity on average but with similar ligand efficiency, and we hypothesized that the inclusion of this information would lead to a better prediction of affinity.[8]

In the current study an updated text mining protocol is presented that annotates orthosteric and allosteric compounds more accurately. Moreover, the data set is additionally curated for class A GPCRs, improving the data set’s quality for this target type. The value of the allosteric annotation of compounds was assessed by training pChEMBL prediction models for class A GPCRs with and without binding type information. Furthermore, models were constructed that were able to predict allosterism of class A GPCR compounds. The entire annotated data set and the script for model training are freely available as a supplement to this article. Additionally, the reader is able to apply text mining on new documents to annotate the binding type of compounds by using the text mining method and keywords that are provided.

Results

Construction of the Orthosteric/Allosteric Binding Type Data Set Using Text Mining

Keyword-enabled text mining was applied on documents retrieved from ChEMBL[7] to determine the binding type of the compounds (see Methods for details). Only the compounds from the assays that were annotated as “primary assay” were categorized as allosteric, orthosteric, bitopic, or covalent. The primary assay was assigned based on the highest number of compounds per assay of each document. In contrast to previous work where an effort was made to characterize allosteric compounds,[8,9] the bitopic and covalent classes were added to remove these compounds from the allosteric and orthosteric classes. Upon manual assessment of the text-mined orthosteric and allosteric compounds for class A GPCRs, it was observed that one document contained a primary assay with computationally derived activity values instead of experimental binding activities.[21] Therefore, compounds from the secondary assay were selected for this document. Furthermore, 46 histamine H4 compounds that were categorized as allosteric were in fact orthosteric, which was detected when checked manually. These compounds were corrected accordingly. To expand the data set, additional “undetermined” class A GPCR documents were manually categorized, which resulted in a further addition of 306 orthosteric and 24 allosteric compounds. For the majority the binding types were well-defined, especially for class A GPCRs, as confirmed by manual random sample assessment and positive model outcomes. The total binding type data set (for all text-mined proteins) contained 201 721 data points (9 390 allosteric, 14 265 orthosteric, and 178 066 undetermined), of which 60 116 defined data points for class A GPCRs (1 442 allosteric, 9 914 orthosteric, and 48 760 undetermined).

Chemical Properties of Allosteric Modulators

The differences between orthosteric and allosteric binders were explored by evaluating the following properties: molecular weight, A Log P, number of hydrogen bond donors and acceptors, activity (pChEMBL), number of rotatable bonds, and number of aromatic bonds. It was observed that the properties of the allosteric modulators for all proteins in ChEMBL were comparable to those of the orthosteric compounds (Table ). In general, the orthosteric compounds covered a wider range of all properties: additional higher molecular weights, more diversity in A Log P, and compounds with more hydrogen bond donors and acceptors. A slight difference was observed in the spread of bioactivities of the two binding types. Although the orthosteric compounds included a higher number of low pChEMBL values (pChEMBL < 4), the bioactivity median was 0.7 log units higher than that of allosteric compounds (pChEMBL 7.1 and 6.4, respectively), in line with previous work.[8] Moreover it was observed that allosteric modulators contained a bigger fraction of aromatic bonds (value normalized to the number of bonds) compared to orthosteric compounds. This corresponds with previous statements that allosteric modulators are more aromatic.[8,9] Please note that for the modeling work below, a more diverse number of descriptors was used. Nonetheless, the physicochemical properties of the allosteric modulators are generally not very different from those of orthosteric compounds.

Table 1

Physicochemical Properties of Orthosteric and Allosteric Compounds in ChEMBLa

	allosteric			orthosteric
	median	MAD	mean and standard deviation	median	MAD	mean and standard deviation
pChEMBL	6.4	0.9	6.5 ± 1.2	7.1	1.0	7.1 ± 1.4
molecular weight	383	61	404 ± 135	434	93	541 ± 474
A Log P	3.8	1.0	3.8 ± 2.0	3.8	1.3	3.1 ± 3.9
num. H donors	1	1	1 ± 2	1	1	4 ± 8
num. H acceptors	4	1	4 ± 2	5	2	6 ± 8
fraction rotatable bonds	0.17	0.05	0.18 ± 0.09	0.20	0.06	0.22 ± 0.12
fraction aromatic bonds	0.50	0.11	0.47 ± 0.17	0.43	0.12	0.42 ± 0.18

The fractions of rotatable bonds and aromatic bonds represent the number of bonds normalized to the total number of bonds per compound. An independent (unpaired) t test gave p < 0.001 (two-tailed, α = 0.05) for the difference between the allosteric and orthosteric means of each property, which indicates a significant difference. MAD = mean absolute deviation.

When only a specific protein class was explored, class A GPCRs in this case, it was observed that the properties of the orthosteric and allosteric compounds were comparable to the mean of compounds for all proteins in ChEMBL. However, we noticed that the properties of individual protein classes could vary significantly compared to the properties of the entire set. Figure (and Supporting Information Figure S1) displays the variances between the ligand properties of different protein classes. Although class A GPCRs contained the second-largest number of mined allosteric modulators (1442 compounds) and are therefore able to cover more chemical ground, it is remarkable that most protein classes have less pronounced properties: the minimum and maximum values for especially molecular weight, A Log P, and hydrogen bond donors and acceptors were less prominent than for the class A GPCRs. The A Log P values of the orthosteric class B and C GPCR compounds (A Log P 5.2 and A Log P −2.7, respectively) were deviating a lot from the conclusion drawn from the entire set where the median A Log P was 3.8. However, these observations could be explained, as shown below.

Figure 2

Physicochemical properties of orthosteric and allosteric compounds in ChEMBL per protein family. Orange boxes indicate the area between the 25th percentile and 50th percentile, gray indicates the area between the 50th percentile and 75th percentile, the border between orange and gray indicates the median value or 50th percentile, and the whiskers indicate the minimum value up to the 25th percentile (bottom whiskers) and the 75th percentile up to the maximum value (top whiskers). (A) pChEMBL value; (B) A Log P; (C) fraction of rotatable bonds; (D) fraction of aromatic bonds.

It was observed that the biggest allosteric set was retrieved for class C GPCRs (2513 compounds), of which the orthosteric site is located in the extracellular Venus flytrap domain, in contrast to other GPCR classes. The orthosteric compounds in this class had a observably lower A Log P than the allosteric modulators: A Log P median orthosteric −2.7 and allosteric 3.7. Given that the orthosteric compounds for class C GPCRs bind to a structurally different binding site than the allosteric modulators for this receptor, this observation is not surprising. Furthermore, it is known that glutamate receptors make up a large fraction of class C GPCRs, for which the main natural ligand is glutamate, a charged amino acid.[22] Hence, orthosteric ligands should resemble these characteristics, leading to the observed shift in A Log P values (the median A Log P of orthosteric compounds binding to glutamate receptors is −3.0). Moreover, it is noteworthy to mention that the class C GPCR compounds might bias the full data set as they represent 27% of the allosteric modulators. A similar trend was observed for class B GPCRs, with an A Log P median for orthosteric compounds of 5.2. It was found that the orthosteric compounds for class B GPCRs consisted of solely peptides and that the allosteric class was only represented by seven modulators, which were also peptides. Therefore, class B GPCRs alone will not be a good case for modeling allosterism.

Class A GPCRs, on the other hand, contained a sufficient number of orthosteric and allosteric compounds (9914 orthosteric and 1442 allosteric) and a well-balanced portion of small molecules and peptides: orthosteric 5428 small molecules, 4395 peptides, and 91 undefined; and allosteric 851 small molecules and 591 peptides (as classified by ChEMBL).[23] It should be noted that the results in this section are based on text-mined binding types. Therefore, caution is advised when comparing and drawing conclusions from the differences observed between orthosteric and allosteric compounds and also between protein classes. Nonetheless, an effort was made to curate the binding types for class A GPCRs with higher accuracy to support the findings in the current study.

Chemical Structures of Class A GPCR Allosteric Modulators

The chemical structures of compounds for the class A GPCRs were explored based on their scaffold trees. The scaffolds were clustered based on two tree levels: the root (smallest scaffold) and the Bemis–Murcko framework[24] (biggest scaffold). Clustering of the class A GPCR set resulted in 313 unique root scaffolds and 4362 Bemis–Murcko scaffolds. The allosteric modulators covered 19 root scaffolds and 564 Bemis–Murcko scaffolds. The orthosteric compounds contained significantly more unique scaffolds: 229 root and 3770 Bemis–Murcko scaffolds. Furthermore, it was observed that some clusters contained both orthosteric and allosteric compounds: 65 root and 28 Bemis–Murcko scaffold clusters were shared. Notably, clusters formed based on the root scaffold are not uniform with the clusters based on Bemis–Murcko scaffolds. In contrast, it was observed that compounds that were grouped into a shared cluster based on root scaffold were clustered into either an orthosteric-specific or an allosteric-specific cluster based on the Bemis–Murcko scaffold (Figure ). This can be explained since the Bemis–Murcko scaffolds are bigger and therefore more specific. The root scaffold, however, is more general and therefore prone to cluster more diverse compounds into the same cluster.

Figure 3

Clustered scaffolds of orthosteric and allosteric compounds for class A GPCRs. Bemis–Murcko scaffolds are depicted with the root scaffold indicated in bold.

Predicting Allosteric Modulation

Convolutional deep neural networks (DNNs, see Methods for network architecture) were applied to observe if allosteric modulation of compounds for class A GPCRs could be predicted. Compounds of the binding type data set that were published before the year 2013 were used for model training. Compounds published in 2013 or later were used for validation. This time-split, or temporal-split, approach has advantages over random-split validation as it gives a more accurate representation of the models’ predictive performances.[25] Furthermore, the training set was balanced to correct for the unequal number of orthosteric and allosteric compounds. This was performed using oversampling of the minority class. The minority class was multiplied until it reached an amount of compounds that was comparable to the majority class.

The allosteric and orthosteric binding types were predicted by a model trained on physicochemical properties, compound fingerprints, and protein descriptors (for details see Methods). The temporal-split validation showed that the proteochemometric (PCM)[26] DNN model, which was trained on compound and protein class A GPCR data, could be used to differentiate allosteric modulators from orthosteric compounds: Matthews correlation coefficient (MCC) = 0.54, sensitivity = 0.54, specificity = 0.94, positive predictive value (PPV) = 0.82, and negative predictive value (NPV) = 0.80. Although sensitivity for allosteric compounds is significantly lower than that for orthosteric compounds (the specificity metric in this case), the model’s predictive ability is sufficient to differentiate between both binding types.

DNNs Outperform Random Forest in Binding Type Predictions

The binding type DNN model was compared to a random forest (RF) model that was trained and tested on exactly the same data (class A GPCRs). The predictive performance of the RF model for allosteric compounds was slightly worse than that of the DNN model with a performance of MCC = 0.43, sensitivity = 0.35, specificity = 0.97, PPV = 0.85, and NPV = 0.74. This result indicates that the DNN model was better than the RF model in identification of the binding type of class A GPCRs.

Enhancement of Bioactivity Predictions by the Incorporation of Binding Types

It was studied if the text mining derived binding types could contribute to the predictive performance of pChEMBL prediction models. Instead of the prediction of binding types, the DNN models were trained to predict the activity of compounds (pChEMBL value ≥6.5 is active), using the binding types as descriptor. The models were validated using the same temporal split as mentioned previously (split on the year 2013). The effect of the implementation of binding types was analyzed by comparison of the performances of the pChEMBL prediction model with and without binding type. The models were trained on both orthosteric and allosteric compounds but validated on allosteric modulators and orthosteric compounds separately to remove any bias that might result from the addition of the binding type descriptors. Potential bias might be induced since the allosteric compounds are on average less potent than the orthosteric compounds: pChEMBL 6.6 ± 1.3 and 7.3 ± 1.3, respectively, for class A GPCRs. The findings for bioactivity predictions for allosteric modulators are reported in this section and Table , and the results for orthosteric compounds are included in Supporting Information File S2. It was observed that the MCC of the class A GPCR pChEMBL prediction model with implemented binding type descriptors was slightly higher than for the pChEMBL prediction model that lacked these descriptors, MCC 0.27 and 0.30, respectively, for allosteric modulators. Remarkably the overall performance (MCC) increased slightly for allosteric bioactivity predictions, but sensitivity decreased (−0.05) when adding binding types. Nevertheless, specificity and accuracy both increased: specificity +0.12 and accuracy +0.03. The receiver operating characteristic (ROC) remained comparable (or slightly worse −0.01).

Table 2

DNN Regression Model Performances for Prediction of Bioactivities of Allosteric Modulators for Class A GPCRsa

training data set	added descriptor	MCC	sensitivity	specificity	accuracy	PPV	NPV	RMSE	ROC
class A GPCRs	–	0.27	0.81	0.44	0.61	0.55	0.74	1.07	0.73
	binding type	0.30	0.74	0.56	0.64	0.58	0.72	1.15	0.72
	predicted binding type	0.22	0.51	0.70	0.62	0.59	0.63	1.01	0.70
GPCRs	–	0.27	0.78	0.48	0.62	0.56	0.73	1.01	0.74
	binding type	0.34	0.78	0.55	0.65	0.59	0.75	1.09	0.73
	predicted binding type	0.15	0.52	0.63	0.58	0.54	0.61	0.99	0.67
all proteins in ChEMBL	–	0.35	0.74	0.61	0.67	0.61	0.74	1.03	0.74
	binding type	0.37	0.74	0.62	0.68	0.62	0.74	0.98	0.75
	predicted binding type	0.18	0.43	0.74	0.60	0.58	0.61	1.06	0.67

MCC = Matthews correlation coefficient, PPV = positive predictive value, NPV = negative predictive value, RMSE = root-mean-square error, and ROC = receiver operating characteristic.

The addition of the binding types and their effect on pChEMBL predictions for allosteric modulators were further evaluated when the model was trained on the full GPCR data set and on the data set containing all proteins in ChEMBL. The pChEMBL predictions for class A GPCRs increased when binding types were included in the models trained on these bigger data sets (Figure ). All metrics improved or performed equally upon the addition of binding types to these enlarged data sets. The pChEMBL predictions for allosteric modulators of class A GPCRs when trained on the full GPCR data set improved the most with a MCC of 0.27 increasing to 0.34. The best performing model for allosteric modulators for class A GPCRs was the PCM model trained on all proteins with binding types included (MCC 0.37).

Figure 4

Performances of DNN regression models for the prediction of bioactivities of class A GPCR allosteric modulators with and without binding type descriptors. MCC = Matthews correlation coefficient, PPV = positive predictive value, and NPV = negative predictive value.

Expansion of Binding Type Predictions into Bioactivity Models

It was observed that the DNN models were able to predict if compounds were binding in an allosteric or orthosteric manner. Furthermore, the addition of binding type descriptors into pChEMBL prediction models enhanced model performance. Therefore, models were applied iteratively by first predicting the binding types of compounds, subsequently followed by implementation of these predictions into training of a pChEMBL prediction model. The compounds used for testing were retrieved from ChEMBL assays that were categorized as “undetermined” by the text mining protocol. The predicted binding types were added as a descriptor which ranged from zero (“0”) to one (“1”), with 1 being allosteric and 0 being orthosteric. The scores for this descriptor were derived as output from the binding type classification model and indicated the probability of a compound belonging to the orthosteric class (score <0.5) and allosteric class (score >0.5). The trained class A GPCR pChEMBL prediction model (regression) was compared to a pChEMBL prediction model that lacked these predicted binding type descriptors. The performance was MCC 0.22, sensitivity 0.51, specificity 0.70, accuracy 0.62, PPV 0.59, NPV 0.63, root-mean-square error 1.01 (RMSE), and ROC 0.70 (Table ). The pChEMBL prediction model that included predicted binding types overall performed worse than the pChEMBL prediction model without the predicted binding types, with the exception of increased specificity and PPV. This may be a result of the implementation of “uncertain” values to generate new predictions; an additional modeling error is introduced into the pChEMBL prediction model that uses predicted descriptors. Furthermore, it was observed that the performances of the models that included predicted binding types decreased when the training set was enlarged with all GPCRs and all proteins in ChEMBL. This is in contrast with the previous observation that model performance increased upon training set enlargement. This indicates that the binding type descriptor weighs heavily in the determination of the pChEMBL value. Therefore, it is of importance that accurate experimentally validated binding types are used in model training, as predicted binding types are not sufficient.

Discussion

Via our text mining protocol we were able to retrieve accurate “orthosteric” and “allosteric” binding types. The addition of these literature mined binding types in pChEMBL prediction models increased model performance. However, an important aspect is that the binding types should be reliable. Implementation of predicted binding types as a descriptor, with a potential error in prediction, did not improve model performance. Although the binding type model did not enhance the accuracy of pChEMBL predictions, the model could be applied to indicate possible allosterism of compounds, as the performance of the binding type model (MCC = 0.54) is remarkably better than random (MCC = 0). In contrast to our study, allosterism predicting models that are reported in the literature are not specified to class A GPCRs or report a less reliable validation method (random split instead of temporal split).[8,27] Therefore, the performance of these models cannot be compared directly.

The physicochemical properties of allosteric modulators generally match the properties of orthosteric compounds; both are druglike.[8] Nevertheless, the chemical features can be distinctive between protein classes, which is reflected by the differences in A Log P, fraction of rotatable bonds, and fraction of aromatic bonds.[9] For class A GPCRs, root and Bemis–Murcko scaffolds were identified that specifically match allosteric modulators or orthosteric compounds. Additionally, scaffolds were found that were shared among allosteric and orthosteric compounds.

Manual evaluation of the text mining results showed that some documents were misclassified. This indicates that the text mining derived binding types are not 100% accurate and therefore might still contain some inaccuracies. However, the majority of the binding types are well-defined, especially for class A GPCRs, as confirmed by manual random sample assessment and positive model outcomes. Furthermore, the addition of specific keywords to filter for covalent and bitopic compounds increases the accuracy of categorization of the allosteric and orthosteric classes. Although addition of a bitopic class in text mining for allosteric modulators has been reported before,[8] implementation of a filter for covalent compounds is only presented in this study. The effort that was made to text mine for binding types has the advantage that multiple binding types can be assembled and annotated. In contrast to the compounds published in the Allosteric Database,[28] which only lists allosteric molecules, orthosteric compounds were also actively annotated in this study and thus available as an additional subset.

The data set that resulted from text mining was not limited to class A GPCRs but contained determined binding types for multiple protein classes in ChEMBL including receptors, enzymes, and solute carriers. Furthermore, some compounds could not be divided into one of the assigned binding type classes, resulting in “undetermined binding type” compounds. These compounds were thus not picked up using the extensive list of keywords. The expansion of the searched text from title and abstract text to full paper text may decrease the number of undetermined compounds. However, feasibility may be difficult as not all papers are open access.

It was shown that the expansion of the training set using all proteins increased pChEMBL predictions for class A GPCRs, a finding that has been observed for PCM modeling previously.[26] Moreover, implementation of the text mining derived binding types increased model performance even further. Previous research shows that the choice of descriptors can influence model performance.[29] Nevertheless, to the best of the authors’ knowledge, the use of binding types as additional descriptors in bioactivity modeling has not yet been reported.

Conclusions

We successfully retrieved allosteric and orthosteric binding types using text mining. Implementation of these binding types into class A GPCR pChEMBL prediction models improved model performance. Although we were able to predict the binding type of compounds, implementation of these predicted binding types into pChEMBL prediction models did not enhance model performance. Since we did observe improvement of model performance when accurate binding types were included, we encourage the use of binding type descriptors in computational models. Unfortunately, binding type information on compounds is not always readily accessible, especially in the public domain. This research therefore stresses the importance of binding type integration in public databases.

Methods

Text Mining and Compound Classification

Binding types, orthosteric or allosteric, were assigned to the compounds using text mining. Documents that contained both a title and an abstract were retrieved from the ChEMBL database (version 22).[23,30,31] The abstracts and titles were screened for keywords to classify the corresponding document in one of the following classes: orthosteric, allosteric, bitopic, and covalent. Documents were classified using a three-layer approach. Remaining, or unclassified, documents were termed “undetermined”. Every layer of the classification process included a different set of keywords (Supporting Information datasheet S3), with the most accurate and general keywords in the first layer and decreased accuracy of keywords in the following layer. The third layer contained general orthosteric keywords that were not applied in the first layer as they were generally only mentioned in conjunction with allosteric keywords. The documents that could not be classified within the first layer continued to the second layer and, if applicable, to the third layer. Every document that contained at least one bitopic or covalent keyword was directly classified into the corresponding class. Classification of orthosteric and allosteric documents was determined by majority vote based on the number of occurrences of keywords belonging to the orthosteric or allosteric class. Compounds from the primary assay, the assay containing the most bioactivities, of the orthosteric and allosteric classified documents were retrieved from ChEMBL and assigned to the same class as the corresponding document. Additionally, class A GPCR compounds from the primary assays of previously undetermined documents were placed in the orthosteric class if published before 2000, as from this year on the literature started more actively reporting allosteric binding types. Therefore, we assumed that the study was performed for orthosteric binders, when allosterism was not explicitly mentioned in a paper in the years prior to 2000. The text mining protocol to categorize documents is included as Supporting Information file S4.

Data Set

The ChEMBL[7] data derived through text mining were filtered on confidence score 5 (multiple direct proteins), 7 (direct protein complex subunits), and 9 (direct single protein). Additionally, values retrieved from PubChem[32] and compounds without a defined or exact pChEMBL value (relation is “=”) were discarded. Only pChEMBL values that were determined using binding assays were kept. Mean pChEMBL values were calculated for bioactivities of duplicate compounds tested on the same target. All compounds were standardized using Pipeline Pilot (version 16.2.0.58):[33] the largest fragment was kept, and stereochemistry and charges were standardized. The data set is included as Supporting Information file S5.

Compound Descriptors

Compounds were encoded by their physicochemical properties (Supporting Information file S6). Additionally FCFP6 fingerprints[34] were used to describe the compounds’ structures. Moreover, the activity types (Ki, Kd, EC50, IC50, and AC50) were added as binary descriptor, with bit “1” if this activity type was used for measuring the activity value of the compound. The nonbinary properties were scaled to values between 0 and 1, where 1 is equal to the highest value obtained for that particular property from ChEMBL. The compound descriptors for all compounds in the data set are included in Supporting Information file S6.

Protein Descriptors

The protein sequences were transformed into descriptors by dividing every sequence into sections that contained five amino acids per section. From these sections the mean values were calculated for the amino acids’ physicochemical properties: molecular weight, number of chiral atoms, Log D, charge, number of hydrogen bonds and acceptors, rigidity, and number of aromatic bonds. By using the physicochemical properties per five amino acids, the protein descriptors become alignment independent (as we have shown previously) allowing us to expand the application area to diverse multiprotein family data sets.[29] Moreover, in this approach descriptor calculation and model training times will be faster compared to the incorporation of physicochemical properties per single amino acid. Additionally, protein taxonomy was taken into account by including classification levels L1, L2, L3, L4, L5, and L6 from ChEMBL.[23] These levels were translated into binary values and added as a protein descriptor.

Model Training

The RF models were trained using the ensemble package from the python Scikit-Learn module.[35] The amount of trees was set to 500 with random split using seed 12345. Convolutional DNN models were trained to predict binding type and pChEMBL values. The multitask DNN models contained two pyramidal shared hidden layers with 2000 and 100 units, respectively. The DNN implementation was built using Python 2.7 on Keras with Theano[36] backend. The following settings were applied: test-size = 0.01 (10%), layer dropout = 0.1, learning rate = 0.0001, momentum = 0.8, patience = 25 s, maximum epochs = 2000, and maximum batch size = 128. The training set was balanced for the binding type classes (orthosteric and allosteric) using oversampling of the underrepresented class. Here, the underrepresented class was multiplied until it reached a corresponding number of entries compared to the opposite class. The number of multiplications was determined by the number of compounds in the majority class divided by the number of compounds in the minority class and rounded off to a whole number. The script to run the RF and DNN models and the configuration file are included as Supporting Information file S7.

Model Validation

Models were validated with a temporal-split approach. The training set consisted of all data from before the year 2013. The test set comprised all data from the publication year 2013 and above. The test set encompassed approximately 15% of the whole class A GPCR data set. The performance of both classification (binding type models) and regression models (pChEMBL prediction models) was evaluated using the metrics MCC, sensitivity, specificity, accuracy, PPV, and NPV.