Why the Data Revolution Needs Qualitative Thinking
by Anissa Tanweer, Emily Kalah Gade, P.M. Krafft, and Sarah Dreier
Published on
Jul 30, 2021
Harvard Data Science Review
https://hdsr.mitpress.mit.edu/pub/u9s6f22y/release/4
In this article, we focus on a set of concepts that are intrinsically informed by particular epistemological and ontological positions common in qualitative social sciences—positions that seek to understand the contingently and subjectively constructed nature of the social world. We refer to these concepts as ‘sensibilities’ because we intend them to intervene on methodology in a sensitizing rather than prescriptive way. The three sensibilities we discuss, have certain kinds of methodological practices, and they can be coupled with multiple modes of data collection and analysis.
Sensibility
Interpretivism
Working definition
An epistemological approach probing the multiple and contingent ways that meaning is ascribed to objects, actions, and situations.
Example of related methods
Trace ethnography (Geiger & Ribes, 2011; Geiger & Halfaker, 2017)
Sensibility
Abductive reasoning
Working definition
A mode of inference that updates and builds upon preexisting assumptions based on new observations in order to generate a novel explanation for a phenomenon.
Example of related methods
Iterations of open coding, theoretical coding, and selective coding (Thornberg & Charmaz, 2013)
Sensibility
Reflexivity
Working definition
A process by which researchers systematically reflect upon their own positions relative to their object, context, and method of inquiry.
Example of related methods
Brain dumps, situational mapping, and toolkit critiques (Markham, 2017)
Abduction
Abduction is often described as “inference to the best explanation” (Douven, 2011).
Abductive reasoning updates and builds upon preexisting assumptions (in other words, theories) based on new observations in order to generate a novel explanation for a phenomenon. As such, it demarks “a creative outcome which engenders a new idea,"
When using abductive reasoning, qualitative researchers have developed ways of addressing the relationships between prior assumptions, new observations, and newly derived explanations. This can be incorporated in data science.
The labeling of data in qualitative methods (what qualitative researchers would instead call ‘coding’) is not a matter of mere assumption, but rather a systematic part of the theory-building process.
LinkedIn AI article
What are the most common reasoning frameworks for data science? Interesting content.
https://www.linkedin.com/advice/0/what-most-common-reasoning-frameworks-data-science-41fif
I made a contribution to the above article*.
The above article in is decision making series
https://www.linkedin.com/pulse/topics/soft-skills-s2976/decision-making-s2506/
From Big Data to Thick Data: Why Qualitative Insights Are the New Gold
Tiffany Perkins-Munn, Ph.D.
Tiffany Perkins-Munn, Ph.D.
C-suite & Board Advisor | Data Science, ML, AI | Global Data Power Women 2025 | Top 100 Most Influential People in Data | AI–100 | ex-BLK, ML, Citadel
https://www.linkedin.com/pulse/from-big-data-thick-why-qualitative-insights-new-gold-tiffany-srqne/
Understanding Qualitative Data and Its Application in Data Science
Zahid Butt
Zahid Butt
Consultancy Services (Data Analyst)
May 16, 2025
https://www.linkedin.com/pulse/understanding-qualitative-data-its-application-science-zahid-butt-iturc/
Jan 11, 2023
AI concepts for beginners - Exploring abductive reasoning in AI
https://indiaai.gov.in/article/exploring-abductive-reasoning-in-ai
Qualitative Data Analysis Methodologies and Methods
Last Updated : 23 Jul, 2025
https://www.geeksforgeeks.org/data-analysis/qualitative-data-analysis-methodologies-and-methods/
Leveraging LLM and Data Science Methods to Quantify Qualitative Data: A Workflow and Method Example
Making Sense of Data: AI-based NLP Tools for Education Research
by Prof.Dr.Ergul Demir | Sep 5, 2025
https://blog.eera-ecer.de/nlp-tools-for-education-research/
20 October 2023
15 Pages
Thematic Analysis of Big Data in Financial Institutions Using NLP Techniques with a Cloud Computing Perspective: A Systematic Literature Review
Full paper
Ratnesh Kumar Sharma, et al.
1
School of Computer Science, Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney 2007, Australia
A Detailed Guide about Natural Language Processing and NLP Techniques Every Data Scientist Should Know
Dejan Markovic
The goal of NLP is to achieve human-level language understanding, allowing software to “read” enormous volumes of documents, analyze sentiment and topics, generate summaries, translate between languages, and even create original text.
some of the most common NLP techniques leveraged today:
Tokenization — Splitting text into sentences and words
Text classification — Categorize and label documents
Stemming — Reduce words to their root form
Sentiment analysis — Detect emotional tone
Keyword extraction — Identify important terms
Semantic search — Understand the meaning and intent behind search queries
Information extraction — Structured data from unstructured text
Language detection — Identify the origin language of the text
Leading NLP Solutions
BERT — Breakthrough NLP model from GoogleAI focusing on language representation
GPT-3 — Advanced language generation model from OpenAI
Amazon Comprehend — AWS service for NLP and natural language understanding
Microsoft LUIS — Azure cognitive service to build language understanding into apps
Powerful Rules-Based Methods
Rules-based NLP techniques apply structured knowledge and logic flows to decipher language:
Parsing involves applying grammar templates and vocabulary to systematically break down sentence constructs and relate words accordingly. This allows proper interpretation of complex linguistics like irony that are often misclassified by statistical models.
Part-of-speech tagging carefully uses lexical rules and dictionaries to classify words as nouns, verbs, adjectives, etc. This structured output provides clarity for downstream processes.
Semantic analysis relies on hierarchical ontologies and semantic networks to encode real-world knowledge and eliminate ambiguity. For instance, determining appropriate synonyms based on a term’s meaning.
Statistical and Neural Machine Learning Methods
Machine learning NLP techniques automatically learn to transform, classify, or simulate language from large datasets:
Text generation models like OpenAI’s celebrated GPT-3 employ multi-layer transformer neural networks trained on enormous unlabeled text corpora to learn contextual relationships within the language and mimic its fluidity with striking fluency. However, their potential for propagating harmful biases due to ingesting entire Internet scrapes remains hotly debated.
Sentence embedding techniques like Google’s BERT representation model set performance records across various language tasks by innovating bidirectional conditioning to capture semantic meaning within phrases for downstream predictive modeling deeply. However, their intense computational requirements yielding an unprecedented 175 billion parameter models have sparked sustainability implications.
Mature sentiment analysis classifiers like VADER leverage logistic regression on top of heuristically defined linguistic rule sets to categorize the effects of diverse text from tweets to lengthy articles to speeches. They now power a plethora of critical applications — from monitoring population mental health to political polling to guiding business
Open access
Research article
First published online November 8, 2023
Natural Language Processing Enhanced Qualitative Methods: An Opportunity to Improve Health Outcomes
R. David Parker https://orcid.org/0000-0002-1955-7445 david.parker@alaska.edu, Karen Mancini, and Marissa D. Abram
Using Natural Language Processing (NLP) to predict the future
Published
November 15, 2025·
Updated
October 13, 2026·
Alex Nikic
https://fastdatascience.com/natural-language-processing/predict-future/
Natural language processing for data analysis: an application on the well-being
Open access
Published: 10 June 2026
Volume 60, pages 18253–18287 (2026)
https://link.springer.com/article/10.1007/s11135-026-02894-9
Bringing Data Science to Qualitative Analysis
Year: 2018, Pages: 325-326
DOI Bookmark: 10.1109/eScience.2018.00076
Authors
You-Wei Cheah, Lawrence Berkeley Nat. Lab., Berkeley, CA, USA
Drew Paine, Lawrence Berkeley Nat. Lab., Berkeley, CA, USA
Devarshi Ghoshal, Lawrence Berkeley Nat. Lab., Berkeley, CA, USA
Lavanya Ramakrishnan, Lawrence Berkeley Nat. Lab., Berkeley, CA, USA
2026 Guide to Generative AI: Techniques, Tools & Trends
https://hatchworks.com/blog/gen-ai/generative-ai/
Methods to Integrate Natural Language Processing Into Qualitative Research
SAGE Publications Inc
International Journal of Qualitative Methods
December 202019(8)
https://www.researchgate.net/publication/347973721_Methods_to_Integrate_Natural_Language_Processing_Into_Qualitative_Research
Review Article
Open Access
The Utilization of Natural Language Processing for Analyzing Social Media Data in Nursing Research: A Scoping Review
Zhenrong Wang, Yulin Ma, Yuanyuan Song, Yao Huang, Guopeng Liang, Xi Zhong
First published: 30 December 2024
https://onlinelibrary.wiley.com/doi/10.1155/jonm/2857497
ud. 8.11.2026
pub. 22.11.2023
