ClimateBERT and NLP for Greenwashing Detection: A Technical Deep Dive

When researchers at ETH Zurich set out to build a language model that could understand climate discourse, they faced a problem that general-purpose models like BERT, GPT, and Llama couldn't solve well: environmental language is highly domain-specific, technically dense, and full of terms that have different meanings in climate contexts than in general usage.

"Net zero" doesn't mean the same thing to a climate scientist as to a marketer. "Transition" in climate policy refers to something fundamentally different than "transition" in a business strategy document. ClimateBERT was built to understand these distinctions — and that capability makes it uniquely suited for greenwashing detection.

Architecture and Training

ClimateBERT is based on the DistilRoBERTa architecture — a distilled (compressed) version of RoBERTa, which is itself an optimised variant of BERT. The choice of DistilRoBERTa over full RoBERTa or BERT-large was pragmatic: it's 40% smaller and 60% faster while retaining 97% of the language understanding capability. For a domain-specific application where inference speed matters, the trade-off is worth it.

The domain adaptation process involved two phases:

Phase 1: Domain-Adaptive Pre-Training (DAPT)

The base DistilRoBERTa model was further pre-trained on a corpus of over 2 million paragraphs (approximately 300 million tokens) from climate-specific sources:

IPCC reports — AR5, AR6, and special reports (technical climate science language)
Corporate sustainability reports — 10K+ reports from CDP, GRI, and company websites (corporate environmental language)
EU regulatory documents — Directives, regulations, guidance notes on environmental policy
Climate research papers — Academic publications on climate science and policy
Financial climate disclosures — TCFD reports, green bond frameworks, ESG fund documents
News articles — Climate and environment journalism from major outlets

This corpus exposes the model to the full spectrum of climate-related language: from rigorous science to regulatory compliance to corporate marketing. The model learns the vocabulary, syntactic patterns, and semantic relationships specific to climate discourse.

Phase 2: Task-Specific Fine-Tuning

After domain pre-training, the model is fine-tuned on specific downstream tasks. For greenwashing detection, this involves training on labelled datasets where human annotators have classified text as greenwashing or genuine environmental communication.

The key fine-tuning datasets include:

ClimaText — 1,000+ annotated paragraphs from corporate reports classified by climate relevance and commitment type
EnvironmentalClaims — Dataset of environmental claims labelled as substantiated, vague, misleading, or false
Net Zero Tracker data — Corporate net-zero pledges with quality assessments
ASA/ARPP rulings — Advertising regulatory decisions on environmental claims, providing ground-truth labels for what constitutes greenwashing

Downstream Tasks for Greenwashing Detection

ClimateBERT has been fine-tuned for several tasks that, combined, create a greenwashing detection pipeline:

Task 1: Climate Text Detection

Binary classification: is this text about climate/environment or not? This serves as the first filter, identifying which parts of a document contain environmental claims that need further analysis.

Benchmark performance: 96.5% F1 score on the ClimaText test set — essentially production-ready for text filtering.

Task 2: Climate Sentiment Analysis

Three-class classification: does this text express climate concern (risk awareness, urgency), climate opportunity (transition benefits, green growth), or climate neutrality (neither positive nor negative)? Greenwashing text tends to over-index on "opportunity" sentiment while under-representing "risk" — a detectable imbalance.

Benchmark performance: 78.4% F1 score. Lower than Task 1 because sentiment is inherently more ambiguous, but sufficient for pattern detection.

Task 3: TCFD Category Classification

Classifies climate-related text into the four TCFD pillars: Governance, Strategy, Risk Management, Metrics & Targets. Greenwashing text tends to concentrate in Strategy and Governance (aspirational language) while avoiding Metrics & Targets (concrete data).

Benchmark performance: 82.1% F1 score across four categories.

Task 4: Commitment Type Classification

Classifies environmental statements as: concrete action ("We installed 5MW of solar in 2025"), measurable target ("We will reduce Scope 1 by 40% by 2030"), vague aspiration ("We are committed to sustainability"), or greenwashing ("Our products are eco-friendly").

Benchmark performance: 86.2% F1 score — the most directly relevant metric for greenwashing detection. This means the model correctly identifies 86 out of 100 greenwashing instances while misclassifying 14.

How Our Scanner Uses NLP

The Green Claims Scanner uses a multi-layer approach inspired by ClimateBERT's architecture but optimised for real-time website scanning:

Web crawling and text extraction: The scanner retrieves page content, strips HTML, and segments text into claim-level units.
Environmental claim detection: A fast classifier identifies which sentences contain environmental claims, filtering out non-relevant content.
Regulatory term matching: Extracted claims are matched against the EU's list of restricted and banned environmental terms, with fuzzy matching to catch variations.
Claim classification: Each identified claim is classified by type (generic, specific, comparative, aspirational) and risk level (high, medium, low).
Risk scoring: An aggregate risk score is calculated based on the number, type, and severity of flagged claims.

The scanner prioritises speed and recall (catching as many potential issues as possible) over precision (avoiding false positives). For a screening tool, missing a real issue is worse than flagging a borderline one — human review handles the grey areas.

Comparing Models: ClimateBERT vs. General Purpose

Model	Greenwashing Detection F1	Climate Sentiment F1	Speed (tokens/sec)
ClimateBERT	86.2%	78.4%	~15,000
RoBERTa-base	74.1%	71.2%	~12,000
BERT-base	71.8%	68.9%	~11,000
GPT-4 (zero-shot)	72.5%	75.1%	~200
Llama-3-8B (zero-shot)	68.3%	70.6%	~800

Two observations stand out. First, domain-specific fine-tuning gives ClimateBERT a 12-14 percentage point advantage over general-purpose models of similar size. Second, large language models (GPT-4, Llama) are competitive on sentiment analysis but underperform on the more structured greenwashing detection task where domain vocabulary precision matters more than general reasoning.

Limitations and Active Research

Language limitation: ClimateBERT was trained primarily on English text. Performance degrades on German, French, and other EU languages. Multilingual variants are in development but not yet at English-level performance.

Temporal knowledge: The model's training data has a cutoff. Post-training regulatory changes (like the ECGT's specific prohibited terms) require periodic retraining or supplementary rule layers.

Implicit greenwashing: The model handles explicit claims well but struggles with implicit greenwashing — where environmental virtue is implied through imagery, narrative structure, or strategic omission rather than specific statements.

Sector calibration: A "low emissions" claim means different things in different industries. ClimateBERT doesn't natively understand sector baselines, requiring supplementary data for sector-appropriate risk assessment.

Practical Applications Beyond Scanning

NLP-based greenwashing detection extends beyond website scanning:

Regulatory monitoring: Authorities can screen thousands of corporate websites systematically
ESG fund analysis: Investors can assess whether fund marketing language matches portfolio composition
Supply chain due diligence: Companies can screen supplier sustainability claims
Pre-publication compliance: Marketing teams can screen content before publication
Academic research: Researchers can analyse greenwashing trends across industries and time periods

The technology is moving from research to production. Within 2-3 years, AI-assisted greenwashing detection will likely be standard regulatory infrastructure across the EU.