Research

The relationships described below were all statistically significant, typically at p < .01. Note that our research (despite being peer-reviewed by PhD experts for quality) is NOT published, to retain our proprietary advantage.

Reliability

The MetaKnowledge™ Index (MKI) used by Sophiometrics is not simply a average of item scores, but a sophisticated synergy of the entire item set (our methodology is unconventional). Thus, for internal consistency, a conventional Cohen’s α does not apply, but bootstrapped simulations (to estimate all possible inter-correlations) are around .75. The MKI technique, applied across different item sets, knowledge domains, question formats, and populations, reliably (across several studies) showed the statistically significant validities described below.

Criterion Validity

A primary outcome variable in our research was undergraduate grade point average (GPA) at an ethnically-diverse, high-ranking university. We chose this because it represented a broad, multi-faceted life outcome, independently tested by multiple experts over significant time reflecting holistic efficacy in a wide spectrum of life challenges for a population representative of emerging generations of professionals. Multiple implementations of the MKI all significantly and distinctly predicted GPA.

Convergent and Discriminant Validity

To better characterize what the MKI represents (and what it doesn’t), we examined several related constructs.

Cognitive abilities: The MKI reliably related to general intelligence, recognition memory accuracy, problem-solving abilities, and clerical accuracy.

Personality: Typical Big Five personality measures did not explain MKI variance, but we did find significant correlations with the six-dimensional HEXACO‘s Honesty-Humility factor and Altruism facet.

Self-Enhancement: The MKI related (negatively) to overconfidence, entitlement, Narcissism, lack of intellectual humility, hindsight bias (“I knew it all along”) and overplacement (“better-than-average” effect). Self-enhancement is effectively a kind of anti-wisdom which may yield short-term gains at the expense of greater long-term losses.

Behaviors: Impatience, cheating, carelessness (as a trait, not just a state), and unjustified ability claims were identified by the MKI.

Metacognition: Self-report measures of metacognitive skills, need for cognition, or growth mindset, did not correlate with the MKI (likely because self-report mixes reality with illusion), but objective behavioral measures (cognitive reflection, overestimation) did.

Demographics: Sex and socio-economic factors did not appear to be significant predictors of the MKI, while cultural variables were inconsistent. However, as expected, the wisdom tracked by the MKI did increase with age.

Incremental Validity

Predicting GPA alone is a low bar, so we controlled for all the measures described above and found the MKI isolated unique variance, significantly increasing R². This is what makes the MKI valuable; it reveals information otherwise overlooked, even by dozens of other measures. In particular, the MKI captures the destructive effects self-enhancement has on the good judgment required for successful knowledge performance, independent of intelligence. The MKI helps you distinguish wise people from the merely smart.

Sample Distributions

Initial development work for the MetaKnowledge Index (MKI) used mostly students from a high-ranking, culturally diverse university. The primary outcome measure was grade-point average (GPA) across all courses and disciplines. In this context, GPA serves as a holistic life “success” indicator because it captures not just intelligence, but also skill managing, time, health, money, relationships, and more, over significant periods, measured independently and extensively by multiple experts.

The following scatter plots show relationships between MKI and GPA (in standard deviations), measured on three different samples with different reference knowledge for the MKI. The correlation statistics indicate sample size (df = N – 2) and 95% confidence interval, all ***p < .001.

The effect size is comparable with general intelligence yet remains significant even when general intelligence and other cognitive and personality factors are controlled for. Statistically, that is the secret sauce, the magic for detecting signals others miss.