This article has Open Peer Review reports available.
Hypothesis exploration with visualization of variance
© Parker et al.; licensee BioMed Central Ltd. 2014
Received: 20 August 2013
Accepted: 2 June 2014
Published: 2 July 2014
The Consortium for Neuropsychiatric Phenomics (CNP) at UCLA was an investigation into the biological bases of traits such as memory and response inhibition phenotypes—to explore whether they are linked to syndromes including ADHD, Bipolar disorder, and Schizophrenia. An aim of the consortium was in moving from traditional categorical approaches for psychiatric syndromes towards more quantitative approaches based on large-scale analysis of the space of human variation. It represented an application of phenomics—wide-scale, systematic study of phenotypes—to neuropsychiatry research.
This paper reports on a system for exploration of hypotheses in data obtained from the LA2K, LA3C, and LA5C studies in CNP. ViVA is a system for exploratory data analysis using novel mathematical models and methods for visualization of variance. An example of these methods is called VISOVA, a combination of visualization and analysis of variance, with the flavor of exploration associated with ANOVA in biomedical hypothesis generation. It permits visual identification of phenotype profiles—patterns of values across phenotypes—that characterize groups. Visualization enables screening and refinement of hypotheses about variance structure of sets of phenotypes.
The ViVA system was designed for exploration of neuropsychiatric hypotheses by interdisciplinary teams. Automated visualization in ViVA supports ‘natural selection’ on a pool of hypotheses, and permits deeper understanding of the statistical architecture of the data. Large-scale perspective of this kind could lead to better neuropsychiatric diagnostics.
Motivation: better neuropsychiatric diagnostics
Diagnosis in neuropsychiatry rests on an elaborate taxonomy of syndromes and explicit decision trees for classification. For example, these decision trees have been codified in DSM-IV  and its very recent revision DSM-V . Dissatisfaction with the current situation has been evident throughout the development of DSM-V, and particularly its inclusion of dimensional classification (quantitative approaches to assessment and diagnosis)  as a step beyond DSM-IV ‘chinese menu’ diagnosis and categories that do not always fit, toward quantitative assessments of severity and treatment response that are grounded in data. However the benefits of dimensional approaches remain controversial. The stakes involved in this evolution are breathtaking, as the DSM is a cornerstone of the mental health system.
Many research efforts are now seeking better models for existing categories such as ADHD, Bipolar Disorder, and Schizophrenia. These rubrics are often said to be inadequate for classification because they rest on inaccurate descriptions. The NIMH Strategic Plan seeks improvements on existing diagnostic categories for mental disorders, both because the categories lack validity and because they limit incorporation of new scientific results. A criticism often leveled against the DSM is that it is not aligned with any scientific model of neuropsychiatric disorders . A related criticism is that different diagnoses overlap significantly, in some cases using different terminology for the same concept.
Large databases can support both statistical evaluation of these criticisms and development of better diagnostics. For example, a recent investigation of differences in disorder incidence rates by gender  focused on patterns of disorder comorbidity based on the very large (n = 43,093) National Epidemiologic Survey on Alcohol and Related Conditions (NESARC). As discussed later, the results offered an overall statistical outline or architecture for disorders, clarifying how they impact men and women differently: women have a higher incidence of internalizing (mood and anxiety) disorders, while men have a higher incidence of externalizing (antisocial and substance use) disorders . This link between gender on disorders has become formalized in the Internalizing-Externalizing ‘meta-structure’ of DSM-V , and the database analysis results in  suggest an important way it can be refined.
A related trend is the development of increasingly sophisticated models based on data . General linear models (GLMs ) or structural equation models (SEMs , often referred to as CSA—covariance structure analysis) are becoming common, permitting characterization of variance structure with a set of functional equations. An emphasis on dimensional models has been developed within the NIMH Research Domains Criteria (RDoC) project [10, 11], particlarly for diagnosis, permitting continuous variables of function, ranging from behavior down to neurobiology. The variance structure models and phenotype profiles emphasized in this paper are consistent with this trend. With large databases, these models have the potential for significant advances in neuropsychiatric diagnosis.
The CNP database
The LA2K study [12, 13] was conducted at UCLA during 2008–2012, designed within CNP as a large-scale analysis for about 2000 volunteers from the Los Angeles metropolitan region. Behind its development was the hypothesis that there may be sufficient variance in healthy people along dimensions shared with people with psychopathology, that we might find common mechanisms with genetic links. For example, the genetic bases for variability in working memory in healthy people may also be the basis for working memory impairments commonly found in patients. LA2K focused on the evaluation of memory and response inhibition as central endophenotypes  with the potential to serve as basic dimensions for neuropsychiatry. Ultimately this approach also could permit development of statistical models characterizing neuropsychiatric syndromes, without relying on traditional taxonomies and their discrete categories.
LA2K was developed in part, then, as a demonstration of the use of phenomics as a framework for neuropsychiatry, and hopefully also a way to advance the pace of research . Phenotypes—detectable or measurable characteristics of an organism—are outward manifestations of interaction of its genotype and environment. Phenomics is the systematic study of biological and behavioral phenotypes . Because phenotypes are present in all scales of science, phenomics is interdisciplinary and intrinsically large-scale in scope. LA2K was a 6-year effort aimed at discovery of relationships that would go undetected with a smaller scope. Initially LA2K obtained data on approximately 1300 subjects (healthy control subjects, not suffering from any major syndrome). The related LA3C and LA5C studies subsequently included some patients diagnosed with ADHD, Bipolar Disorder, and Schizophrenia, increasing opportunities for discovery.
Tasks and Tests in the LA2K database
CNP database domain/Test
Consent/Screening/Diagnosis/Clinical Rating Scales
Adult ADHD Interview (module from KSADS-PL)
Hopkins Symptom Checklist 25
Structured Clinical Interview for DSM-IV/Axis I/Patient version
The Temperament and Character Inventory
The Chapman Scales – Physical Anhedonia
The Chapman Scales – Social Anhedonia
The Chapman Scales – Perceptual Aberrations
Eckblad and Chapman’s Hypomanic Personality Scale
Golden and Meehl’s 7-Item Schizoid Scale
Munich Chronotype Questionnaire
Akiskal’s Bipolar II Scale
Barratt Impulsivity Scale
Eysenck’s Impulsivity Venturesome and Empathy Inventory
MPQ (Control-Impulsivity items)
The Dickman Scale of Functional vs Dysfunctional Impulsivity
Spatial and Verbal Memory and Manipulation Tasks
Spatial and Verbal Working Memory Capacity Tasks
Scene Recognition Task
California Verbal Learning Test
WMS-III Spatial Span
WMS-III Digit Span
WMS-III Visual Reproduction (part I/II [immed/delayed recall])
WMS-III Letter Number Sequencing
Conner’s CPT II
Task Set Switching
Attention Networks Task
Balloon Analog Risk Task
In general terms, the CNP database permits investigators to analyze the behavior of subjects in neurocognitive measures that reflect memory and response inhibition (recording reaction times and accuracy measures), as well as aspects of temperament, personality, and syndromal behavior. Each table in Table 1 has data for the relevant subjects.
The LA2K study was designed around a schema spanning 7 levels of neuroscience, from genome to syndrome, centering on memory and response inhibition phenotypes . The levels were designed to facilitate development of new models for syndromes, with hypotheses based on these phenotypes.
The dendrograms in Figure 1 show block structure, and suggest how the variables could be partitioned into clusters representing dimensions of significant variance. These clusters may correspond to phenotype profiles—patterns of values across sets of phenotypes that characterize important categories. A benefit of using a phenomics approach is that we may be able to identify useful profiles, and also be able to explain the variance in the population with models. For example, methods like PCA  might be used to summarize the variables in this matrix with about 20 dimensions, since the first 20 components explain about 50% of its variance.
Variance structure models
Variance structure is a common term in data analysis. It typically refers to patterns of variation in statistical models related to distributions (variance of individual variables), covariance and correlation of variables, and more general equational relationships among variables. A variance structure model is a mathematical expression of these relationships.
Important classes of variance structure models include general linear models (GLMs ) and structural equation models (SEMs). These permit characterization of variance structure in terms of a set of functional equations that are often linear in form. However they permit nonlinear interactions (nested models) and conditioning on non-numeric variables or factors that take discrete values (multi-level models).
Variance structure is often exposed by an incremental process of decomposition or factoring, yielding a hierarchy or graph of components that together form an overall model. For example, PCA is a linear algebraic model of covariance structure. From the standpoint of decomposition, this process also can differentiate clusters of similar variables from others, and extract hierarchical block structure.
An important aspect of variance structure is that it can often be visualized. For example, it is not a coincidence that the hierarchical model foundation for the popular trellis graphics visualization framework  corresponds to grids of graphics that highlight variation. This insight was the spark that led to the ViVA (Visualization of VAriance) system architecture described in this paper.
a subset S of the data (a population of subjects that is meaningful for analysis).
a grouping G, specifying a set of class names (factor levels) defining several subpopulations.
zero or more dependent variables Y from the CNP database. (If zero, all variables are independent).
one or more independent variables X from the CNP database.
a model of variance M (a mathematical model and/or visualization method for explaining variance).
relates reaction time to age groups that define population subsets—in this case restricted to control subjects. Multi-level models with this kind of conditioning are basic to ANOVA, explaining variance across the different values of a factor, such as the Female and Male values of Gender.
Although ViVA supports development of hypotheses about the study data, it deliberately has no facilities for traditional statistical hypothesis testing. This resolves a tension between discovery and hypothesis testing. A recent review  contrasts ‘-ology’ strategies that typically test a priori hypotheses with ‘-omics’ strategies that adopt more agnostic, exploratory approaches, stressing that “... discovery-based approaches do not eschew hypotheses; rather, they seek to elevate hypothesis testing to a new level, by allowing high-throughput hypothesis generation and prioritization”.
ViVA focuses on exploration of variance structure hypotheses in a large phenotype database. Exploration of hypotheses can involve analytical generation and prioritization, but a particular strength of ViVA has been as a source of ‘natural selection’ in a hypothesis pool. Grounding hypotheses in data—checking them as assertions about the CNP database—permits screening of weaker hypotheses and refinement of hypotheses that survive. With this kind of testing, the process of hypothesis generation  becomes one of evolution.
Objectives of this paper
This paper describes ViVA’s support for exploring hypotheses. The system was designed for neuropsychiatric hypothesis exploration by interdisciplinary teams, and more specifically for grounding them in data (checking them as concrete assertions about LA2K). Automatically-constructed visualizations in ViVA permit exploration and deeper understanding of variance structure, and grounding improves the hypothesis pool. With a sequence of examples, we illustrate how this approach can contribute towards the development of better neuropsychiatric diagnostics.
The system is novel in several ways. First, ViVA directly links visualization with variance structure models, permitting interactive visual exploration of natural models in a large database. It includes VISOVA, for example, a combination of visualization and analysis of variance with the exploratory flavor that is associated with ANOVA in biomedical hypothesis generation. Viewing variance structure models as hypotheses, it provides automated support for integrative analysis of large phenotype databases. The examples given attempt to illustrate its potential in developing greater understanding of phenotypic variation.
This project initially began as a server aimed at sharing of exploratory data analysis scripts for the CNP database. These evolved, until it eventually became clear that they emphasized visual analogues of ANOVA—including what we call VISOVA—holding some variables fixed, varying others, and permitting visualization of the resulting ‘response’.
ViVA Atlas—‘gallery’ of results for predefined hypotheses.
ViVA Viewer—simple menu-based hypothesis exploration.
ViVA Explorer—advanced development of hypotheses.
These three differ mainly in the power of the interface. The Atlas has a simple interface, allowing immediately access to wide sets of results, with large pre-computed reports that overview the data. The Viewer allows relation-level exploration, permitting rapid visual exploration of entire tables listed in Table 1. The Explorer has an advanced interface, permitting definition of population groups and analysis of specific variables. These three systems can be used in sequence, progressing from initial passive orientation to active exploration.
The architecture common to these three systems aims at making exploration as effortless as possible. This boils down to automation: automated data cleaning, implementation of group tables, creation of tables and fields wherever needed, development of web infrastructure, and introduction of features aimed at simplifying interaction and rising above the complexity of the database.
implementing visualization with a server has advantages for pooling effort (such as in data cleaning and visualization scripts), and for maintaining best practices and standards (such as verification of distributions).
using universally-understood vocabularies of visualization and exploratory data analysis (EDA  and ANOVA) can help in trans-disciplinary work, particularly among scientists with little programming experience.
Hypotheses in ViVA Explorer can involve any subset of 2500 phenotype variables, and can condition on any of 60 group structures in 18 predefined population subsets (as well as on all experimental protocols). This permits very flexible definition of variance structure models. Results of analyses are ‘web sites’ that can be refined or extended collaboratively at any time.
Visualization of variance
Our variance structure models M :Y∼X∣G:(S) are adaptations of the formulas Y∼X∣G supported by linear models in the S and R statistical computing environments . They are also a mainstay of trellis graphics, an influential visualization toolkit in these environments. In trellis graphics, the values (factor levels) of conditioning variables define an array or grid (‘trellis’) of similar visualizations, allowing side-by-side comparison across these values. Thus the conditioning variables yield automatic construction of grids of visualizations.
Side-by-side display is a powerful tool for visualization of variance, although we have not seen it presented or developed in this way. A new emphasis of ViVA is in directly linking visualization with variance structure models, permitting interactive visual exploration of dimensional representations of disorder.
How does ViVA differ from existing visualization systems? For example, the Clinical suite of tools in Spotfire , a widely-used visualization framework, provides summary statistics, relation plots, and many other types of charts. Most statistical computing environments provide these functions as well. Although it doesn’t provide more functions or analytical power than these systems, ViVA is novel and has some strengths.
ViVA goes to lengths to make exploration effortless. It consists of about 50,000 lines of Python code, which is equally split between web interface and back-end data management. The latter involves automation of updates in a data extraction and cleaning pipeline, implementation of group structures and subpopulations, creation of useful extra tables and fields, and visualization functions that have been improved over time. These things can be implemented in any environment, but LA2K is complex, and ViVA provides three exploration modes (Atlas, Viewer, Explorer), which led to a custom design.
Existing research practice is very different—it begins with a basic hypothesis and concludes with statistical hypothesis tests. Generally the hypothesis concerns a small set of variables, involving at most a few tables, with questions like those behind the design of LA2K. The first step is specific (and deliberately limited) selection of variables from the data dictionary, followed by data download, manual cleaning and reformatting as spreadsheets, and loading into a statistical computing environment. Visualization is not heavily used if at all, partly out of principle and partly because of the perceived energy expenditure required to generate useful displays. Statistical computing environments provide trustworthy hypothesis testing.
Although ViVA includes no hypothesis testing, the time required to complete the other research steps—from selection of populations and variables to obtaining analysis reports—is at most a few minutes. The resulting visualizations immediately detect mistakes in variable selection and identify concerns about variable distributions and assumptions. Visualization can instantly suggest discarding or refining a hypothesis. ViVA cannot automate human hypothesis development, but it can support it with exploration.
Results and discussion
We illustrate hypothesis exploration in ViVA—with visualization of variance—with several example scenarios. These applications highlight points that motivated the development of LA2K, such as the value of phenotype profiles that cut across disciplines. However the main thing they illustrate is how exploration can refine hypotheses. Although it cannot validate or prove hypotheses, then, ViVA can ground them in data—and introduce forces of natural selection that yield better hypotheses. This section offers a few examples to illustrate what ViVA has to offer for hypothesis development and visualization of variance. More examples are available at http://www.hypweb.org.
Sample hypothesis: age and reaction time
Consider a rough hypothesis that reaction time (RT) is affected by age. We can explore this hypothesis by considering all Mean RT values (joined across tables). Figure 3 shows a VISOVA (parallel coordinates/ANOVA) visualization for a table containing all key MEAN RT variables, essentially a sequence of variance structure models V I S O V A : ∼R e a c t i o n T i m e∣A g e:(L A 2K c o n t r o l), in which each column shows the effect of age on one (independent) RT variable. This axis labels shows all of the MEAN RT variables included and the columns representing variables are clustered according to correlation similarity. In other words, each column represents the value range of a single variable, and trajectories across the columns give the sequence of RT values for a single subject.
VISOVA displays extend parallel coordinates with group structure. Individual group averages for the variables are also superimposed as thicker lines along with standard error bars; the colors reflect the 7 age ranges used as groups. In other words, the continuous age variable here has been used to obtain discrete population subgroups. The thick red line at the top shows the average for subjects aged 41–50, while the blue line near the bottom shows the average for subjects aged 21–30. Thus a thick line represents a phenotype profile (pattern of average values) for a particular group. The gradual increase of RT over these age groups is consistent across all Mean RT variables, suggesting an ongoing process of decline (progressive increase in mean reaction time) with age, regardless of task or type of RT measure (e.g., congruent vs. incongruent RT). This continuum in this effect suggests that a mathematical model of decline might be possible.
The recent article  notes that Age is often incorrectly treated as a ‘nuisance variable’—a quantity of little perceived interest or relevance, yet which must be statistically controlled as it might indirectly affect quantities of interest. It asserts that controlling for age can give distorted views of disease processes; many psychosocial factors are influenced by the stage of life. Nuisance assumptions are difficult to check, but this example shows that doing it can be imperative, and visualization can help.
The results here encourage exploration of still stronger hypotheses. For example, the same variance structure model could be specialized with other factors—such as demographic factors like gender or ethnicity, and behavioral factors like smoking habits. Any of the 60 factors in ViVA could be checked. Changes to the hypothesis above to incorporate new group structure or include other variables only require changing selections in the interaction menu. The interface encourages exploration of assumptions.
Sample hypothesis: timing and schizophrenia
Hypotheses regarding the characterization of Schizophrenia in terms of interval timing have emerged recently . Various deficits in temporal processing are associated with symptoms of the disorder. It is interesting to explore whether LA2K can provide some support for this association.
The database includes a large number of temporal phenotypes, many in the form of reaction time (RT) measurements. These RT variables include mean and standard deviation (a digest of multiple trials), but provide nothing further for interpretation. Consequently, ViVA was constructed to augment these with both SNR (signal-to-noise-ratio =μ/σ) and CV (coefficient of variation =σ/μ) variables, normalizations that permit useful visual comparison of values. In other words, ViVA augments the database variables with useful related measures of variance.
ViVA can explore hypotheses about differences in RT variability (RT CV) exhibited in Schizophrenia. An advantage of RT CV instead of RT alone is that it is less sensitive to the underlying reaction time distribution. Some related published work is in , including arguments for using CV in measuring reaction time. Although there has been recent interest in response time variability as a measure, there is almost no existing work relating RT variability and Schizophrenia.
Figure 4 shows a VISOVA display of CV values that cut across the database, with the variance structure model V I S O V A : ∼R e a c t i o n T i m e C V∣G:(L A 2K c o n t r o l+L A 3C p a t i e n t) where G is an ad hoc group constructed to include control subjects and Schizophrenia patients. In this plot, Schizophrenia patients have uniformly higher CV values, and the differences exceed the standard error. This cannot validate hypotheses, but it is a demonstration of how one can ground hypotheses in data. It might encourage exploring other measures of variability—such as RT SNR and RT FF (Fano factor =σ 2 /μ).Exploration of variance structure—with ViVA’s automatic augmentation of the database with CV variables—in this case suggests new hypotheses, with interesting results. Despite the broad set of RT measures in Figure 4 (all CV measures in the CNP database—spanning SMNM, DDT, ANT, SCWT, SR, and SST), with few exceptions the differences are greater than the standard error (error bars). Furthermore, more exploration suggested that RT CV is not clearly age-dependent (unlike MEAN RT in Figure 3), encouraging investigation of RT CV as a supplementary measure to RT.
Sample hypotheses: ethnicity profiles
Next we offer two examples in which ViVA raised doubt about hypotheses. Both involve ethnicity.
The health phenotypes in Figure 5 shows no differences between Hispanic and non-Hispanic subjects. To put this in perspective, it is not hard to find ethnic differences that could have links to health. Figure 5 shows clear differences for BMI, so variance structure is evident. However, this variance does not extend to the LA2K Health Score—a sum across 21 items asking about diagnoses with, or treatment for, a number of serious medical illnesses. In other words, no pattern was apparent in C o v a r i a n c e :H e a l t h∼B M I∣E t h n i c i t y:(L A 2K c o n t r o l), and exploration did not find any health differences linked to ethnicity.
There are other interesting hypotheses regarding ethnicity that are based in genetics. For example, specific variants of the DRD4 gene have been associated with Novelty Seeking (NS) behavior, and a well-known hypothesis is that NS was important for human migration out of Africa 50,000 years ago . A careful recent analysis confirmed an association between the DRD4 2R and 7R polymorphisms and migratory distance. The DRD4 7R polymorphism also has been associated with a variety of phenotypes related to NS, including ADHD and smoking phenotypes. However, there is controversy about these associations, and differing results have emerged in populations from different countries [.An initial hypothesis then might be that subjects of Hispanic ethnicity have higher NS scores. Figure 5 shows that grounding this hypothesis in LA2K suggests the opposite: NS-related summary variables from the TCI yield lower score values for Hispanic subjects. Furthermore, smoking measures were lower for Hispanic subjects. This profile appears robust, consistent across multiple indicators.
However, the LA2K ‘ethnicity’ classification is self-reported, and controls were screened for ADHD. For clarity the ethnicity of subjects was constrained to match both parents in Figure 5. The measures related to Novelty Seeking from the TCI inventory were consistently higher for non-Hispanic subjects. We found support only for negative hypotheses V I S O V A : ∼N S∣E t h n i c i t y:(L A 2K c o n t r o l) relating NS to Hispanic ethnicity. Grounding of both ethnicity hypotheses in data raised only doubts.
Development of ethnicity profiles as a robust form of hypothesis checking is an interesting direction for work with complex phenotypes like NS and health. Furthermore, the eventual incorporation of genome-wide genetic variation data in ViVA will allow investigators to stratify groups based on more precise measures of ancestral background, as opposed to relying solely on self-reported ethnicity. Generally, phenotype profiles fit the phenomics/cross-disciplinary outlook mentioned earlier. Multi-population, multi-phenotype views also have found success before in integrative visualization systems -.
Hypotheses about neuropsychiatric statistical architecture: gender and disorder prevalence
As mentioned earlier, large databases and systems like ViVA may become important for developing better neuropsychiatric diagnostics. A noteworthy example is in differences in disorder incidence rates by gender reported in , which analyzed patterns of comorbidity in the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC, n>40,000). The results not only clarify how disorders affect men and women differently, but also offer an overall statistical architecture for this difference.
Recently a dimensional model of comorbidity among these disorders appeared in , with the surprising claim that the comorbidity prevalence of disorders is identical in both genders. That is, once the latent gender-specific liability levels are conditioned for, the structure of common disorders is gender-invariant. This result suggests a single overarching disorder structure. It also bypasses a basic problem in analyzing comorbidity—the assumption that the categories of the disorders are valid. If the disorders are dimensional, comorbidity should be dimensional also.
Reported levels of disorder prevalence by gender have varied , although the outline of internalizing and externalizing liability has been confirmed. These confirmations were important in the arduous DSM-V development process, which has required years of deliberations and field trials. The ability to explore the structure of human variation in large databases like NESARC, with systems like ViVA, could be a way to increase consensus and incorporate scientific models into neuropsychiatric practice.
Phenomics and phenotype databases are natural settings for hypothesis exploration. In biomedicine, hypotheses often concern variance structure—patterns of variation in variables when group and population structure are controlled. The abundance of extensions for ANOVA in the statistical and biomedical literature  show the importance of variance structure. However ViVA is novel in combining it with visualization as a system, introducing methods like VISOVA to integrate visualization and ANOVA.
In ViVA, a variance structure model is an assertion M : Y∼X∣G:(S) controlling variables G in population S, representing variance M among the variables Y and X. For example, a model such as VISOVA : Reaction Time∼Age∣Gender:(L A 2K c o n t r o l) represents changes in reaction time by age across healthy male and female populations. There should be significant difference in the association between variables X when conditioned on the different groups.
Multi-level models are important in neuropsychiatry for many reasons, including their basic connections with nosology and diagnosis. Among these, a fundamental aspect of conditioning is that it makes hypotheses differential—they consider not only a base hypothesis with a single level value, but also alternative level values. For example, a hypothesis could assert that phenotype profiles differ characteristically for each group (i.e., level). Without this differential structure, hypotheses are difficult to falsify, and difficult to make mutually exclusive. As a result they are difficult to verify or contradict, and they all can be ‘right’. This lack of exclusivity, and the difficulty of grounding hypotheses in data, impedes progress .
ViVA provides a way, even for scientists with little programming experience, to ‘try hypotheses on for size’ by grounding them in data. It permits selection of any subset of the 2500 database variables, conditioning on any of 60 group structures in 18 predefined populations (as well as on all experimental protocols), using any of a large set of standard variance visualization schemes, without concerns about implementation or details of data cleaning (because these steps are provided by ViVA). To avoid confusion about scientific validity of the results, it is also intentionally limited to hypothesis exploration without hypothesis testing. The exploration process is one of rapid evolution under selection, with stronger hypotheses surviving.
Amidst the deluge of data in which scientists now find themselves, it is vital to integrate relevant information with complex hypotheses. ViVA is an example of ways science can expand from established hypothesis-based processes to more data-driven, discovery-based processes that benefit from the abundance of information.
This research supported by NIH grants RL1LM009833 (Hypothesis Web), UL1DE019580 (UL1RR024911 – UCLA Consortium for Neuropsychiatric Phenomics (CNP)), P20RR020750, and P20MH065166.
- American Psychological Association: DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, IV). 2007, [http://www.psych.org/MainMenu/Research/DSMIV.aspx],Google Scholar
- American Psychological Association: DSM-V (Diagnostic and Statistical Manual of Mental Disorders, V). 2014, [http://www.dsm5.org/],Google Scholar
- Regier D:Dimensional approaches to psychiatric classification refining the research agenda for DSM-V: an introduction. Int J Methods Psychiatr Res. 2007, 16 Suppl 1: S1-S5.View ArticlePubMedGoogle Scholar
- Mclaren N: Humanizing Madness. 2007, Ann Arbor: Loving Healing Press,Google Scholar
- Eaton N, Keyes K, Krueger R, Balsis S, Skodol A, Markon K, Grant B, Hasin D:An invariant dimensional liability model of gender differences in mental disorder prevalence: evidence from a national sample. J Abnorm Psychol. 2012, 121: 282-288.View ArticlePubMedGoogle Scholar
- Hartung C, Widiger T:Gender differences in the diagnosis of mental disorders: conclusions and controversies of the DSM-IV. Psychol Bull. 1998, 123 (3): 260-278.View ArticlePubMedGoogle Scholar
- Simms L, Prisciandaro J, Krueger R, Goldberg D:The structure of depression, anxiety and somatic symptoms in primary care. Psychol Med. 2012, 42: 15-28. 10.1017/S0033291711000985.View ArticlePubMedGoogle Scholar
- Chambers R, Hastie T: Statistical Models in S, (Ch.2: Statistical models). 1992, Independence: Wadsworth/Brooks-Cole (Cengage Learning),Google Scholar
- Shipley W: Cause and Correlation in Biology. 2000, London: Cambridge University Press,View ArticleGoogle Scholar
- NIMH:Research Domains Criteria (RDoC), Draft 3.1. 2011,Google Scholar
- Morris S, Cuthbert B:Research Domain Criteria: cognitive systems, neural circuits, and dimensions of behavior. Dialogues Clin Neurosci. 2012, 14: 29-37.PubMedPubMed CentralGoogle Scholar
- LA2K. 2007, [http://www.phenomics.ucla.edu/LA2K%20Flier_121609.pdf],
- Bilder R, Sabb F, Cannon T, London E, Jentsch J, Parker D, Poldrack R, Evans C, Freimer N:Phenomics: the systematic study of phenotypes on a genome-wide scale. Neuroscience. 2009, 164: 30-42. 10.1016/j.neuroscience.2009.01.027.View ArticlePubMedPubMed CentralGoogle Scholar
- Insel T, Cuthbert B:Endophenotypes: bridging genomic complexity and disorder heterogeneity. Biol Psychiatry. 2009, 66 (11): 988-989. 10.1016/j.biopsych.2009.10.008.View ArticlePubMedGoogle Scholar
- Bilder RM:Neuropsychology 3.0: evidence-based science and practice. J Int Neuropsychol Soc. 2011, 17 (1): 7-13. 10.1017/S1355617710001396.View ArticlePubMedGoogle Scholar
- Thomas R, Dougherty M, Sprenger A, Harbison J:Diagnostic hypothesis generation and human judgment. Psychol Rev. 2008, 115: 155-185.View ArticlePubMedGoogle Scholar
- Pendergrass S, Dudek S, Crawford D, Ritchie M:Synthesis-View: visualization and interpretation of SNP association results for multi-cohort, multi-phenotype data and meta-analysis. BioData Min. 2010, 3: 10-10.1186/1756-0381-3-10.View ArticlePubMedPubMed CentralGoogle Scholar
- Pendergrass S, Dudek S, Crawford D, Ritchie M:Visually integrating and exploring high throughput Phenome-Wide Association Study (PheWAS) results using PheWAS-View. BioData Min. 2012, 5: 5-10.1186/1756-0381-5-5.View ArticlePubMedPubMed CentralGoogle Scholar
- González I, Cao K, Davis M, Déjean S:Visualising associations between paired ‘omics’ data sets. BioData Min. 2012, 5: 19-10.1186/1756-0381-5-19.View ArticlePubMedPubMed CentralGoogle Scholar
- Joliffe I: Principal Components Analysis. 1986, New York: Springer-Verlag,View ArticleGoogle Scholar
- Cleveland W: Visualizing Data. 1993, Independence: Wadsworth/Brooks-Cole (Cengage Learning),Google Scholar
- Geschwind D, Konopka G:Neuroscience in the era of functional genomics and systems biology. Nature. 2009, 461 (7266): 908-915. 10.1038/nature08537.View ArticlePubMedPubMed CentralGoogle Scholar
- Tukey J: Exploratory Data Analysis. 1977, Reading: Addison-Wesley,Google Scholar
- Tibco:Spotfire for Clinical applications. 2011, [http://spotfire.tibco.com/en/discover-spotfire/who-uses-spotfire/by-industry/life-sciences-pe.aspx]],Google Scholar
- Tibco:Spotfire Analytics — Transforming Clinical Development. 2011, [http: //spotfire.tibco.com/media/content-center/whitepapers/spotfire-clinical-development.ashx],Google Scholar
- Sanderman R, Coyne J, Ranchor A:Age: nuisance variable to be eliminated with statistical control or important concern?. Patient Educ Couns. 2006, 61 (3): 315-316. 10.1016/j.pec.2006.04.002.View ArticlePubMedGoogle Scholar
- Ward R, Kellendonk C, Kandel E, Balsam P:Timing as a window on cognition in schizophrenia. Neuropharmacology. 2012, 62 (3): 1175-1181. 10.1016/j.neuropharm.2011.04.014.View ArticlePubMedGoogle Scholar
- Narayanan N, Land B, Solder J, Deisseroth K, DiLeone R:Prefrontal D1 dopamine signaling is required for temporal control. Proc Natl Acad Sci U S A. 2012, 109 (50): 20726-20731. 10.1073/pnas.1211258109.View ArticlePubMedPubMed CentralGoogle Scholar
- Der G, Deary I:Age and sex differences in reaction time in adulthood: results from the United Kingdom Health and Lifestyle Survey. Psychol Aging. 2006, 21: 62-73.View ArticlePubMedGoogle Scholar
- Wagenmakers E, Brown S:On the linear relation between the mean and the standard deviation of a response time distribution. Psychol Rev. 2007, 114 (3): 830-841.View ArticlePubMedGoogle Scholar
- Watson CT, Steinberg KM, Huddleston J, Warren RL, Malig M, Schein J, Willsey AJ, Joy JB, Scott JK, Graves TA, Wilson RK, Holt RA, Eichler EE, Breden F:Complete haplotype sequence of the human immunoglobulin heavy-chain variable, diversity, and joining genes and characterization of allelic and copy-number variation. Am J Hum Genet. 2013, 92 (4): 530-546. 10.1016/j.ajhg.2013.03.004.View ArticlePubMedPubMed CentralGoogle Scholar
- Matthews L, Butler P:Novelty-seeking DRD4 polymorphisms are associated with human migration distance out-of-Africa after controlling for neutral population gene structure. Am J Phys Anthropol. 2011, 145 (3): 382-389. 10.1002/ajpa.21507.View ArticlePubMedGoogle Scholar
- Matthews L, Butler P:Novelty-seeking DRD4 polymorphisms are associated with human migration distance out-of-Africa after controlling for neutral population gene structure. Am J Phys Anthropol. 2011, 145 (3): 382-389. 10.1002/ajpa.21507.View ArticlePubMedGoogle Scholar
- Reist C, Ozdemir V, Wang E, Hashemzadeh M, Mee S, Moyzis R:Novelty seeking and the dopamine D4 receptor gene (DRD4) revisited in Asians: haplotype characterization and relevance of the 2-repeat allele. Am J Med Genet B Neuropsychiatr Genet. 2007, 144B (4): 453-457. 10.1002/ajmg.b.30473.View ArticlePubMedGoogle Scholar
- Hettema J, Prescott C, Myers J, Neale M, Kendler K:The structure of genetic and environmental risk factors for anxiety disorders in men and women. Arch Gen Psychiatry. 2005, 62 (2): 182-189. 10.1001/archpsyc.62.2.182.View ArticlePubMedGoogle Scholar
- Kendler K, Myers J:The boundaries of the internalizing and externalizing genetic spectra in men and women. Psychol Med. 2014, 44 (3): 647-655. 10.1017/S0033291713000585.View ArticlePubMedGoogle Scholar
- Cannon T:What is the role of theories in the study of schizophrenia?. Schizophr Bull. 2009, 35 (3): 563-567. 10.1093/schbul/sbp008.View ArticlePubMedPubMed CentralGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.