Pub Date : 2025-06-25DOI: 10.1177/00491241251336794
Oscar Stuhler, Cat Dang Ton, Etienne Ollion
Generative AI (GenAI) is quickly becoming a valuable tool for sociological research. Already, sociologists employ GenAI for tasks like classifying text and simulating human agents. We point to another major use case: the extraction of structured information from unstructured text. Information Extraction (IE) is an established branch of Natural Language Processing, but leveraging the affordances of this paradigm has thus far required familiarity with specialized models. GenAI changes this by allowing researchers to define their own IE tasks and execute them via targeted prompts. This article explores the potential of open-source large language models for IE by extracting and encoding biographical information (e.g., age, occupation, origin) from a corpus of newspaper obituaries. As we proceed, we discuss how sociologists can develop and evaluate prompt architectures for such tasks, turning codebooks into “promptbooks.” We also evaluate models of different sizes and prompting techniques. Our analysis showcases the potential of GenAI as a flexible and accessible tool for IE while also underscoring risks like non-random error patterns that can bias downstream analyses.
{"title":"From Codebooks to Promptbooks: Extracting Information from Text with Generative Large Language Models","authors":"Oscar Stuhler, Cat Dang Ton, Etienne Ollion","doi":"10.1177/00491241251336794","DOIUrl":"https://doi.org/10.1177/00491241251336794","url":null,"abstract":"Generative AI (GenAI) is quickly becoming a valuable tool for sociological research. Already, sociologists employ GenAI for tasks like classifying text and simulating human agents. We point to another major use case: the extraction of structured information from unstructured text. Information Extraction (IE) is an established branch of Natural Language Processing, but leveraging the affordances of this paradigm has thus far required familiarity with specialized models. GenAI changes this by allowing researchers to define their own IE tasks and execute them via targeted prompts. This article explores the potential of open-source large language models for IE by extracting and encoding biographical information (e.g., age, occupation, origin) from a corpus of newspaper obituaries. As we proceed, we discuss how sociologists can develop and evaluate prompt architectures for such tasks, turning codebooks into “promptbooks.” We also evaluate models of different sizes and prompting techniques. Our analysis showcases the potential of GenAI as a flexible and accessible tool for IE while also underscoring risks like non-random error patterns that can bias downstream analyses.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"20 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144479192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-20DOI: 10.1177/00491241251341196
Martin Nybom, Jan Stuhler
Using complete-count register data spanning three generations, we document spatial patterns in inter- and multi-generational mobility in Sweden. Across municipalities, grandfather–child correlations in education or earnings tend to be larger than the square of the parent–child correlations, suggesting that the latter understate status transmission in the long run. Yet, conventional parent–child correlations capture regional differences in long-run transmission and therefore remain useful for comparative purposes. We further find that the within-country association between mobility and income inequality (the “Great Gatsby Curve”) is at least as strong in the multi- as in the inter-generational case. Interpreting those patterns through the lens of a latent factor model, we find that regional differences in mobility primarily reflect variation in the transmission of latent advantages, rather than in how those advantages translate into observed outcomes.
{"title":"Geographic Variation in Multigenerational Mobility","authors":"Martin Nybom, Jan Stuhler","doi":"10.1177/00491241251341196","DOIUrl":"https://doi.org/10.1177/00491241251341196","url":null,"abstract":"Using complete-count register data spanning three generations, we document spatial patterns in inter- and multi-generational mobility in Sweden. Across municipalities, grandfather–child correlations in education or earnings tend to be larger than the square of the parent–child correlations, suggesting that the latter understate status transmission in the long run. Yet, conventional parent–child correlations capture regional <jats:italic>differences</jats:italic> in long-run transmission and therefore remain useful for comparative purposes. We further find that the within-country association between mobility and income inequality (the “Great Gatsby Curve”) is at least as strong in the multi- as in the inter-generational case. Interpreting those patterns through the lens of a latent factor model, we find that regional differences in mobility primarily reflect variation in the transmission of latent advantages, rather than in how those advantages translate into observed outcomes.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"175 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144328662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-20DOI: 10.1177/00491241251347982
Deirdre Bloome
Researchers concerned about intergenerational inequalities study absolute and relative mobility (e.g., whether people’s adult incomes exceed their parents’ incomes in dollars or ranks ). Absolute and relative mobility are connected, by definition. Yet, they are not equivalent. Indeed, they often diverge. To illuminate why, when, and for whom such divergence occurs—and why, when, and for whom convergence is possible—this article provides two frameworks for connecting absolute and relative mobility. One framework is formal and one is typological. Both frameworks center micro-level socioeconomic experiences across generations. Illustrative analyses employ these frameworks using National Longitudinal Survey of Youth data. Results suggest that divergent experiences, like upward absolute mobility despite downward relative mobility, may be more common among more advantaged social groups. Future researchers could use the two frameworks introduced here to further advance our understanding of how intergenerational inequalities evolve differently in absolute and relative terms.
{"title":"Absolute and Relative Mobility: Two Frameworks for Connecting Intergenerational Mobility in Absolute and Relative Terms","authors":"Deirdre Bloome","doi":"10.1177/00491241251347982","DOIUrl":"https://doi.org/10.1177/00491241251347982","url":null,"abstract":"Researchers concerned about intergenerational inequalities study <jats:italic>absolute</jats:italic> and <jats:italic>relative</jats:italic> mobility (e.g., whether people’s adult incomes exceed their parents’ incomes in <jats:italic>dollars</jats:italic> or <jats:italic>ranks</jats:italic> ). Absolute and relative mobility are connected, by definition. Yet, they are not equivalent. Indeed, they often diverge. To illuminate why, when, and for whom such divergence occurs—and why, when, and for whom convergence is possible—this article provides two frameworks for connecting absolute and relative mobility. One framework is formal and one is typological. Both frameworks center micro-level socioeconomic experiences across generations. Illustrative analyses employ these frameworks using National Longitudinal Survey of Youth data. Results suggest that divergent experiences, like upward absolute mobility despite downward relative mobility, may be more common among more advantaged social groups. Future researchers could use the two frameworks introduced here to further advance our understanding of how intergenerational inequalities evolve <jats:italic>differently</jats:italic> in absolute and relative terms.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"7 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144328660","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-19DOI: 10.1177/00491241251347983
Xi Song, Xiang Zhou
Social mobility scholars have long been interested in estimating the effect of intergenerational mobility, typically measured by differences in the socioeconomic status between parents and offspring, on later-life outcomes of offspring. In a 2022 article “Heterogeneous Effects of Intergenerational Social Mobility: An Improved Method and New Evidence,” Luo proposes a new approach called the mobility contrast model (MCM) to define and estimate mobility effects. We argue that the MCM is inherently flawed due to its reliance on the coding scheme used for the categorical variables of social origin and destination. Specifically, when different coding schemes are applied, the estimands defined in the MCM bear distinct meanings, involve different but equally arbitrary constraints, and sometimes yield contradictory results. Moreover, regardless of the coding scheme, these estimands do not adequately capture the sociological concept of a mobility effect. To illustrate this, we reanalyze the Occupational Changes in a Generation Study data used in Luo’s study, highlighting the inconsistency of results when dummy coding versus effect coding schemes are used.
{"title":"Is There a Mobility Effect? On Methodological Issues in the Mobility Contrast Model","authors":"Xi Song, Xiang Zhou","doi":"10.1177/00491241251347983","DOIUrl":"https://doi.org/10.1177/00491241251347983","url":null,"abstract":"Social mobility scholars have long been interested in estimating the effect of intergenerational mobility, typically measured by differences in the socioeconomic status between parents and offspring, on later-life outcomes of offspring. In a 2022 article “Heterogeneous Effects of Intergenerational Social Mobility: An Improved Method and New Evidence,” Luo proposes a new approach called the mobility contrast model (MCM) to define and estimate mobility effects. We argue that the MCM is inherently flawed due to its reliance on the coding scheme used for the categorical variables of social origin and destination. Specifically, when different coding schemes are applied, the estimands defined in the MCM bear distinct meanings, involve different but equally arbitrary constraints, and sometimes yield contradictory results. Moreover, regardless of the coding scheme, these estimands do not adequately capture the sociological concept of a mobility effect. To illustrate this, we reanalyze the Occupational Changes in a Generation Study data used in Luo’s study, highlighting the inconsistency of results when dummy coding versus effect coding schemes are used.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"19 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144328666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-19DOI: 10.1177/00491241251347984
Haowen Zheng, Siwei Cheng
How well can individuals’ parental background and previous life experiences predict their mid-life socioeconomic status (SES) attainment? This question is central to stratification research, as a strong power of earlier experiences in predicting later-life outcomes signals substantial intra- or intergenerational status persistence, or put simply, social rigidity. Running machine learning models on panel data to predict outcomes that include hourly wage, total income, family income, and occupational status, we find that a large number (around 4,000) of predictors commonly used in the stratification literature improves the prediction of one’s life chances in middle to late adulthood by about 10 percent to 50 percent, compared with a null model that uses a simple mean of the outcome variable. The level of predictability depends on the specific outcome being analyzed, with labor market indicators like wages and occupational prestige being more predictable than broader socioeconomic measures such as overall personal and family income. Grouping a comprehensive list of predictors into four unique sets that cover family background, childhood and adolescence development, early labor market experiences, and early adulthood family formation, we find that including income, employment status, and occupational characteristics at early career significantly improves models’ prediction accuracy for mid-life SES attainment. We also illustrate the application of the predictive models to examine heterogeneity in predictability by race and gender and identify important variables through this data-driven exercise.
{"title":"Social Rigidity Across and Within Generations: A Predictive Approach","authors":"Haowen Zheng, Siwei Cheng","doi":"10.1177/00491241251347984","DOIUrl":"https://doi.org/10.1177/00491241251347984","url":null,"abstract":"How well can individuals’ parental background and previous life experiences predict their mid-life socioeconomic status (SES) attainment? This question is central to stratification research, as a strong power of earlier experiences in predicting later-life outcomes signals substantial intra- or intergenerational status persistence, or put simply, social rigidity. Running machine learning models on panel data to predict outcomes that include hourly wage, total income, family income, and occupational status, we find that a large number (around 4,000) of predictors commonly used in the stratification literature improves the prediction of one’s life chances in middle to late adulthood by about 10 percent to 50 percent, compared with a null model that uses a simple mean of the outcome variable. The level of predictability depends on the specific outcome being analyzed, with labor market indicators like wages and occupational prestige being more predictable than broader socioeconomic measures such as overall personal and family income. Grouping a comprehensive list of predictors into four unique sets that cover family background, childhood and adolescence development, early labor market experiences, and early adulthood family formation, we find that including income, employment status, and occupational characteristics at early career significantly improves models’ prediction accuracy for mid-life SES attainment. We also illustrate the application of the predictive models to examine heterogeneity in predictability by race and gender and identify important variables through this data-driven exercise.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"51 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144319669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-02DOI: 10.1177/00491241251338412
Ian Lundberg, Daniel Molitor, Jennie E. Brand
To what degree does parent occupation cause a child’s occupational attainment? We articulate this causal question in the potential outcomes framework. Empirically, we show that adjustment for only two confounding variables substantially reduces the estimated association between parent and child occupation in a U.S. cohort. Methodologically, we highlight complications that arise when the treatment variable (parent occupation) can take many categorical values. A central methodological hurdle is positivity: some occupations (e.g., lawyer) are simply never held by some parents (e.g., those who did not complete college). We show how to overcome this hurdle by reporting summaries within subgroups that focus attention on the causal quantities that can be credibly estimated. Future research should build on the longstanding tradition of descriptive mobility research to answer causal questions.
{"title":"The Causal Effect of Parent Occupation on Child Occupation: A Multivalued Treatment with Positivity Constraints","authors":"Ian Lundberg, Daniel Molitor, Jennie E. Brand","doi":"10.1177/00491241251338412","DOIUrl":"https://doi.org/10.1177/00491241251338412","url":null,"abstract":"To what degree does parent occupation cause a child’s occupational attainment? We articulate this causal question in the potential outcomes framework. Empirically, we show that adjustment for only two confounding variables substantially reduces the estimated association between parent and child occupation in a U.S. cohort. Methodologically, we highlight complications that arise when the treatment variable (parent occupation) can take many categorical values. A central methodological hurdle is positivity: some occupations (e.g., lawyer) are simply never held by some parents (e.g., those who did not complete college). We show how to overcome this hurdle by reporting summaries within subgroups that focus attention on the causal quantities that can be credibly estimated. Future research should build on the longstanding tradition of descriptive mobility research to answer causal questions.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"245 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144193171","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-02DOI: 10.1177/00491241251337316
Austin C. Kozlowski, James Evans
Large language models (LLMs), through their exposure to massive collections of online text, learn to reproduce the perspectives and linguistic styles of diverse social and cultural groups. This capability suggests a powerful social scientific application—the simulation of empirically realistic, culturally situated human subjects. Synthesizing recent research in artificial intelligence and computational social science, we outline a methodological foundation for simulating human subjects and their social interactions. We then identify six characteristics of current models that are likely to impair the realistic simulation of human subjects: bias, uniformity, atemporality, disembodiment, linguistic cultures, and alien intelligence. For each of these areas, we discuss promising approaches for overcoming their associated shortcomings. Given the rate of change of these models, we advocate for an ongoing methodological program for the simulation of human subjects that keeps pace with rapid technical progress, and caution that validation against human subjects data remains essential to ensure simulation accuracy.
{"title":"Simulating Subjects: The Promise and Peril of Artificial Intelligence Stand-Ins for Social Agents and Interactions","authors":"Austin C. Kozlowski, James Evans","doi":"10.1177/00491241251337316","DOIUrl":"https://doi.org/10.1177/00491241251337316","url":null,"abstract":"Large language models (LLMs), through their exposure to massive collections of online text, learn to reproduce the perspectives and linguistic styles of diverse social and cultural groups. This capability suggests a powerful social scientific application—the simulation of empirically realistic, culturally situated human subjects. Synthesizing recent research in artificial intelligence and computational social science, we outline a methodological foundation for simulating human subjects and their social interactions. We then identify six characteristics of current models that are likely to impair the realistic simulation of human subjects: bias, uniformity, atemporality, disembodiment, linguistic cultures, and alien intelligence. For each of these areas, we discuss promising approaches for overcoming their associated shortcomings. Given the rate of change of these models, we advocate for an ongoing methodological program for the simulation of human subjects that keeps pace with rapid technical progress, and caution that validation against human subjects data remains essential to ensure simulation accuracy.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"113 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144210944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-02DOI: 10.1177/00491241251340080
Hannah Waight, Solomon Messing, Anton Shirikov, Margaret E. Roberts, Jonathan Nagler, Jason Greenfield, Megan A. Brown, Kevin Aslett, Joshua A. Tucker
How can one understand the spread of ideas across text data? This is a key measurement problem in sociological inquiry, from the study of how interest groups shape media discourse, to the spread of policy across institutions, to the diffusion of organizational structures and institution themselves. To study how ideas and narratives diffuse across text, we must first develop a method to identify whether texts share the same information and narratives, rather than the same broad themes or exact features. We propose a novel approach to measure this quantity of interest, which we call “narrative similarity,” by using large language models to distill texts to their core ideas and then compare the similarity of claims rather than of words, phrases, or sentences. The result is an estimand much closer to narrative similarity than what is possible with past relevant alternatives, including exact text reuse, which returns lexically similar documents; topic modeling, which returns topically similar documents; or an array of alternative approaches. We devise an approach to providing out-of-sample measures of performance (precision, recall, F1) and show that our approach outperforms relevant alternatives by a large margin. We apply our approach to an important case study: The spread of Russian claims about the development of a Ukrainian bioweapons program in U.S. mainstream and fringe news websites. While we focus on news in this application, our approach can be applied more broadly to the study of propaganda, misinformation, diffusion of policy and cultural objects, among other topics.
{"title":"Quantifying Narrative Similarity Across Languages","authors":"Hannah Waight, Solomon Messing, Anton Shirikov, Margaret E. Roberts, Jonathan Nagler, Jason Greenfield, Megan A. Brown, Kevin Aslett, Joshua A. Tucker","doi":"10.1177/00491241251340080","DOIUrl":"https://doi.org/10.1177/00491241251340080","url":null,"abstract":"How can one understand the spread of ideas across text data? This is a key measurement problem in sociological inquiry, from the study of how interest groups shape media discourse, to the spread of policy across institutions, to the diffusion of organizational structures and institution themselves. To study how ideas and narratives diffuse across text, we must first develop a method to identify whether texts share the same information and narratives, rather than the same broad themes or exact features. We propose a novel approach to measure this quantity of interest, which we call “narrative similarity,” by using large language models to distill texts to their core ideas and then compare the similarity of <jats:italic>claims</jats:italic> rather than of words, phrases, or sentences. The result is an estimand much closer to narrative similarity than what is possible with past relevant alternatives, including exact text reuse, which returns lexically similar documents; topic modeling, which returns topically similar documents; or an array of alternative approaches. We devise an approach to providing out-of-sample measures of performance (precision, recall, F1) and show that our approach outperforms relevant alternatives by a large margin. We apply our approach to an important case study: The spread of Russian claims about the development of a Ukrainian bioweapons program in U.S. mainstream and fringe news websites. While we focus on news in this application, our approach can be applied more broadly to the study of propaganda, misinformation, diffusion of policy and cultural objects, among other topics.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"62 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144210939","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-02DOI: 10.1177/00491241251345463
Yongchao Ma, Nino Mushkudiani, Barry Schouten
In a probability sampling survey, adaptive data collection strategies may be used to obtain a response set that minimizes nonresponse bias within budget constraints. Previous research has stratified the target population into subgroups defined by categories of auxiliary variables observed for the entire population, and tailored strategies to obtain similar response rates across subgroups. However, if the auxiliary variables are weakly correlated with the target survey variables, optimizing data collection for these subgroups may not reduce nonresponse bias and may actually increase the variance of survey estimates. In this paper, we propose a stratification method to identify subgroups by: (1) predicting values of target survey variables from auxiliary variables, and (2) forming subgroups with different response propensities based on the predicted values of target survey variables. By tailoring different data collection strategies to these subgroups, we can obtain a response set with less variation in response propensities across subgroups that are directly relevant to the target survey variables. Given this rationale, we also propose to measure nonresponse bias by the coefficient of variation of response propensities estimated from the predicted target survey variables. A case study using the Dutch Health Survey shows that the proposed stratification method generally produces less variation in response propensities with respect to the predicted target survey variables compared to traditional methods, thereby leading to a response set that better resembles the population.
{"title":"An Optimal Stratification Method for Addressing Nonresponse Bias in Bayesian Adaptive Survey Design","authors":"Yongchao Ma, Nino Mushkudiani, Barry Schouten","doi":"10.1177/00491241251345463","DOIUrl":"https://doi.org/10.1177/00491241251345463","url":null,"abstract":"In a probability sampling survey, adaptive data collection strategies may be used to obtain a response set that minimizes nonresponse bias within budget constraints. Previous research has stratified the target population into subgroups defined by categories of auxiliary variables observed for the entire population, and tailored strategies to obtain similar response rates across subgroups. However, if the auxiliary variables are weakly correlated with the target survey variables, optimizing data collection for these subgroups may not reduce nonresponse bias and may actually increase the variance of survey estimates. In this paper, we propose a stratification method to identify subgroups by: (1) predicting values of target survey variables from auxiliary variables, and (2) forming subgroups with different response propensities based on the predicted values of target survey variables. By tailoring different data collection strategies to these subgroups, we can obtain a response set with less variation in response propensities across subgroups that are directly relevant to the target survey variables. Given this rationale, we also propose to measure nonresponse bias by the coefficient of variation of response propensities estimated from the predicted target survey variables. A case study using the Dutch Health Survey shows that the proposed stratification method generally produces less variation in response propensities with respect to the predicted target survey variables compared to traditional methods, thereby leading to a response set that better resembles the population.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"51 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144210943","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-05-27DOI: 10.1177/00491241251339673
Tina Law, Elizabeth Roberto
Although there is growing social science research examining how generative AI models can be effectively and systematically applied to text-based tasks, whether and how these models can be used to analyze images remain open questions. In this article, we introduce a framework for analyzing images with generative multimodal models, which consists of three core tasks: curation, discovery, and measurement and inference. We demonstrate this framework with an empirical application that uses OpenAI's GPT-4o model to analyze satellite and streetscape images ( n = 1,101) to identify built environment features that contribute to contemporary residential segregation in U.S. cities. We find that when GPT-4o is provided with well-defined image labels, the model labels images with high validity compared to expert labels. We conclude with thoughts for other use cases and discuss how social scientists can work collaboratively to ensure that image analysis with generative multimodal models is rigorous, reproducible, ethical, and sustainable.
{"title":"Generative Multimodal Models for Social Science: An Application with Satellite and Streetscape Imagery","authors":"Tina Law, Elizabeth Roberto","doi":"10.1177/00491241251339673","DOIUrl":"https://doi.org/10.1177/00491241251339673","url":null,"abstract":"Although there is growing social science research examining how generative AI models can be effectively and systematically applied to text-based tasks, whether and how these models can be used to analyze images remain open questions. In this article, we introduce a framework for analyzing images with generative multimodal models, which consists of three core tasks: curation, discovery, and measurement and inference. We demonstrate this framework with an empirical application that uses OpenAI's GPT-4o model to analyze satellite and streetscape images ( <jats:italic>n</jats:italic> = 1,101) to identify built environment features that contribute to contemporary residential segregation in U.S. cities. We find that when GPT-4o is provided with well-defined image labels, the model labels images with high validity compared to expert labels. We conclude with thoughts for other use cases and discuss how social scientists can work collaboratively to ensure that image analysis with generative multimodal models is rigorous, reproducible, ethical, and sustainable.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"58 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144153930","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：