Skip to content
DataFlowBench

Current overview of DataFlowBench

Each kernel panel keeps one analyzer within one language's own denominator. Its bar label and confusion-matrix dialog list decisive, inconclusive, unsupported, and runner-error counts separately; unsupported alone is a declared decline.

Each kernel poses one positive and one negative assertion per semantic template, and each language's kernel is its own population — cores of 28, 31, 33, 34 or 35 templates, never pooled or ranked against each other, and never compared with the smaller cores of an earlier snapshot. Because every pair is balanced, an analyzer that always answers the same way — or answers blindly where it cannot see a construct — scores exactly half on the affected pairs: read correctness against that 50% blind baseline, and read approximation character from each vendor's TPR/FPR split in the details dialog. A blank analyzer on a kernel means no report in this freeze: no extractor, no frontend, or no adapter, which is coverage rather than a score.

DataFlowBench is SlopCop's own benchmark for semantic data flow: we build it to measure where Bifrost stands against 7 reference analyzers, and to keep ourselves honest while doing it. The methodology is deliberately analyzer-neutral — every number here is generated from digest-bound freeze evidence that we cannot edit after the fact, misses and crashes included — but the motivation is not neutral, and we would rather say so.

What v0.9.2 contains

This fixed-configuration snapshot records the v0.9.2 release model and its retained outcomes. Bifrost was measured at v0.15.1. Inconclusive results preserve incomplete proof; unsupported results record declared coverage; runner errors record failed executions. None becomes a negative finding. CodeQL's native Java, JavaScript, and Python profiles include query-pack lookup runner errors, documented in the case-variation note. The generated figures beside this text come from this snapshot's model.

Below the kernels sit two further populations: the modeling matrix, which asks whether each tool's own model-declaration surface is load-bearing, and the tool-native probes, which ask what each product decides with nothing supplied by us. No certified latency corpus is published for these pins. A separate diagnostic timing supplement records candidate observations without changing the frozen results. The v0.9.0 latency page remains bound to its older pins. All populations have separate denominators. There is no combined leaderboard, and benchmark-controlled results are never pooled with, or compared number-to-number against, tool-native ones.

On the direct-flow breadth baseline, Bifrost answered both assertions correctly in 13 of 13 languages.

Frozen case results, across three populations
4,252
Kernel languages, 28, 31, 33, 34 or 35 templates each
14
Languages in the direct-flow breadth baseline
13
Immutable evidence release
v0.9.2

How decisive, and how correct

Overall analyzer decisiveness and correctness Scatter plot for Overall. The horizontal axis is decisive answers divided by 952 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 952 assertions answered Correctness among decisive answers Bifrost CodeQL Joern Semgrep CE OpenTaint Infer FlowDroid Pysa

Overall decisiveness uses all 952 assertions in the 13 kernel corpus as every analyzer's denominator; a language with no analyzer entry therefore remains visible as unanswered coverage.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnanswered
Bifrost 528/952 (55.5%) 527/528 (99.8%) 424
CodeQL 740/952 (77.7%) 594/740 (80.3%) 212
Joern 350/952 (36.8%) 274/350 (78.3%) 602
Semgrep CE 154/952 (16.2%) 132/154 (85.7%) 798
OpenTaint 140/952 (14.7%) 121/140 (86.4%) 812
Infer 194/952 (20.4%) 157/194 (80.9%) 758
FlowDroid 140/952 (14.7%) 122/140 (87.1%) 812
Pysa 70/952 (7.4%) 56/70 (80.0%) 882
c analyzer decisiveness and correctness Scatter plot for c. The horizontal axis is decisive answers divided by 56 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 56 assertions answered Correctness among decisive answers Bifrost CodeQL Semgrep CE Infer

c is its own 56-assertion population. Analyzers without a c kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 42/56 (75.0%) 42/42 (100.0%) 14
CodeQL 55/56 (98.2%) 47/55 (85.5%) 1
Semgrep CE 14/56 (25.0%) 12/14 (85.7%) 42
Infer 56/56 (100.0%) 47/56 (83.9%) 0
cpp analyzer decisiveness and correctness Scatter plot for cpp. The horizontal axis is decisive answers divided by 68 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 68 assertions answered Correctness among decisive answers Bifrost CodeQL Semgrep CE Infer

cpp is its own 68-assertion population. Analyzers without a cpp kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 30/68 (44.1%) 30/30 (100.0%) 38
CodeQL 67/68 (98.5%) 50/67 (74.6%) 1
Semgrep CE 14/68 (20.6%) 12/14 (85.7%) 54
Infer 68/68 (100.0%) 53/68 (77.9%) 0
csharp analyzer decisiveness and correctness Scatter plot for csharp. The horizontal axis is decisive answers divided by 70 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 70 assertions answered Correctness among decisive answers Bifrost CodeQL

csharp is its own 70-assertion population. Analyzers without a csharp kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 44/70 (62.9%) 43/44 (97.7%) 26
CodeQL 69/70 (98.6%) 55/69 (79.7%) 1
go analyzer decisiveness and correctness Scatter plot for go. The horizontal axis is decisive answers divided by 70 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 70 assertions answered Correctness among decisive answers Bifrost CodeQL Semgrep CE

go is its own 70-assertion population. Analyzers without a go kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 44/70 (62.9%) 44/44 (100.0%) 26
CodeQL 69/70 (98.6%) 54/69 (78.3%) 1
Semgrep CE 14/70 (20.0%) 12/14 (85.7%) 56
java analyzer decisiveness and correctness Scatter plot for java. The horizontal axis is decisive answers divided by 70 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 70 assertions answered Correctness among decisive answers Bifrost CodeQL Joern Semgrep CE OpenTaint Infer FlowDroid

java is its own 70-assertion population. Analyzers without a java kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 46/70 (65.7%) 46/46 (100.0%) 24
CodeQL 70/70 (100.0%) 57/70 (81.4%) 0
Joern 70/70 (100.0%) 57/70 (81.4%) 0
Semgrep CE 14/70 (20.0%) 12/14 (85.7%) 56
OpenTaint 70/70 (100.0%) 61/70 (87.1%) 0
Infer 70/70 (100.0%) 57/70 (81.4%) 0
FlowDroid 70/70 (100.0%) 61/70 (87.1%) 0
javascript analyzer decisiveness and correctness Scatter plot for javascript. The horizontal axis is decisive answers divided by 70 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 70 assertions answered Correctness among decisive answers Bifrost CodeQL Joern Semgrep CE

javascript is its own 70-assertion population. Analyzers without a javascript kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 48/70 (68.6%) 48/48 (100.0%) 22
CodeQL 70/70 (100.0%) 57/70 (81.4%) 0
Joern 70/70 (100.0%) 55/70 (78.6%) 0
Semgrep CE 14/70 (20.0%) 12/14 (85.7%) 56
kotlin analyzer decisiveness and correctness Scatter plot for kotlin. The horizontal axis is decisive answers divided by 70 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 70 assertions answered Correctness among decisive answers Bifrost CodeQL Semgrep CE OpenTaint FlowDroid

kotlin is its own 70-assertion population. Analyzers without a kotlin kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 32/70 (45.7%) 32/32 (100.0%) 38
CodeQL 70/70 (100.0%) 55/70 (78.6%) 0
Semgrep CE 14/70 (20.0%) 12/14 (85.7%) 56
OpenTaint 70/70 (100.0%) 60/70 (85.7%) 0
FlowDroid 70/70 (100.0%) 61/70 (87.1%) 0
php analyzer decisiveness and correctness Scatter plot for php. The horizontal axis is decisive answers divided by 70 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 70 assertions answered Correctness among decisive answers Bifrost Joern Semgrep CE

php is its own 70-assertion population. Analyzers without a php kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 34/70 (48.6%) 34/34 (100.0%) 36
Joern 70/70 (100.0%) 58/70 (82.9%) 0
Semgrep CE 14/70 (20.0%) 12/14 (85.7%) 56
python analyzer decisiveness and correctness Scatter plot for python. The horizontal axis is decisive answers divided by 70 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 70 assertions answered Correctness among decisive answers Bifrost CodeQL Joern Semgrep CE Pysa

python is its own 70-assertion population. Analyzers without a python kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 42/70 (60.0%) 42/42 (100.0%) 28
CodeQL 69/70 (98.6%) 56/69 (81.2%) 1
Joern 70/70 (100.0%) 58/70 (82.9%) 0
Semgrep CE 14/70 (20.0%) 12/14 (85.7%) 56
Pysa 70/70 (100.0%) 56/70 (80.0%) 0
ruby analyzer decisiveness and correctness Scatter plot for ruby. The horizontal axis is decisive answers divided by 70 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 70 assertions answered Correctness among decisive answers Bifrost CodeQL Joern Semgrep CE

ruby is its own 70-assertion population. Analyzers without a ruby kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 36/70 (51.4%) 36/36 (100.0%) 34
CodeQL 69/70 (98.6%) 57/69 (82.6%) 1
Joern 70/70 (100.0%) 46/70 (65.7%) 0
Semgrep CE 14/70 (20.0%) 12/14 (85.7%) 56
rust analyzer decisiveness and correctness Scatter plot for rust. The horizontal axis is decisive answers divided by 62 assertions. The vertical axis is correct answers divided by decisive answers, from 0% to 100%; a line marks the 50% a blind answerer would score. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 62 assertions answered Correctness among decisive answers Bifrost CodeQL Joern Semgrep CE

rust is its own 62-assertion population. Analyzers without a rust kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 42/62 (67.7%) 42/42 (100.0%) 20
CodeQL 62/62 (100.0%) 49/62 (79.0%) 0
Joern 0/62 (0.0%) 0/0 (0.0%) 62
Semgrep CE 14/62 (22.6%) 12/14 (85.7%) 48
scala analyzer decisiveness and correctness Scatter plot for scala. The horizontal axis is decisive answers divided by 70 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 70 assertions answered Correctness among decisive answers Bifrost

scala is its own 70-assertion population. Analyzers without a scala kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 42/70 (60.0%) 42/42 (100.0%) 28
swift analyzer decisiveness and correctness Scatter plot for swift. The horizontal axis is decisive answers divided by 66 assertions. The vertical axis is correct answers divided by decisive answers, from 0% to 100%; a line marks the 50% a blind answerer would score. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 66 assertions answered Correctness among decisive answers CodeQL Joern

swift is its own 66-assertion population. Analyzers without a swift kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
CodeQL 0/66 (0.0%) 0/0 (0.0%) 66
Joern 0/66 (0.0%) 0/0 (0.0%) 66
typescript analyzer decisiveness and correctness Scatter plot for typescript. The horizontal axis is decisive answers divided by 70 assertions. The vertical axis is correct answers divided by decisive answers, from 40% to 100%; a line marks the 50% a blind answerer would score. 40% 50% 60% 70% 80% 90% 100% 0% 25% 50% 75% 100% blind baseline Decisiveness: share of 70 assertions answered Correctness among decisive answers Bifrost CodeQL Semgrep CE

typescript is its own 70-assertion population. Analyzers without a typescript kernel entry are absent, never plotted as zero.

Exact figures
AnalyzerDecisivenessCorrectness when decisiveUnansweredEvidence
Bifrost 46/70 (65.7%) 46/46 (100.0%) 24
CodeQL 70/70 (100.0%) 57/70 (81.4%) 0
Semgrep CE 14/70 (20.0%) 12/14 (85.7%) 56

Confusion matrix · DataFlowBench v0.9.2

Decisive outcomes only. inconclusive preserves incomplete proof, unsupported records a declared capability boundary, and runner-error records a failed execution. All are excluded from the matrix and rates; none becomes a clean negative.

What this result means

DataFlowBench measures whether analyzers correctly decide semantic data-flow questions — and whether they stay quiet when they should. Cases are balanced positive/negative pairs of language-neutral semantic templates (aliasing, kills, call context, branch joins, exception paths, …). The kernels and the modeling matrix run under a benchmark-controlled model profile, so the engines are compared under a common contract; the tool-native probes run under a tool-native profile, measuring the shipped product instead. The two profiles answer different questions and are never combined.

This snapshot's bounded claim covers the synthetic direct-flow breadth baseline and the propagation kernels of the 14 kernel languages above on the taint track, plus the modeling matrix and the tool-native probe set in java, javascript, and python. Cores are sized per language (28, 31, 33, 34 or 35 templates), so each kernel is read on its own denominator; language-only constructs are reported in separate language-extension tiers on the snapshot pages. It does not estimate real-project accuracy or other languages' kernel behavior; performance never enters these scores. Raw durations remain in retained execution evidence, without a published latency corpus or certified current timing comparison. The tool-native rows describe shipped coverage on six probe templates rather than product accuracy at large. inconclusive preserves incomplete proof, unsupported records declared coverage, and runner-error records failed execution; none becomes a clean negative.

Accuracy and language coverage — current snapshot

Every analyzer is shown in the same field. Farther right means documented data-flow support for more languages; higher means more correct assertions within the kernels the analyzer covers. A specialist can therefore show its accuracy without hiding the cost of its narrower language support. The horizontal axis is an analyzer capability, not a count of the adapters DataFlowBench happens to implement. Benchmark kernel participation remains visible in the exact figures below. These are two independent dimensions, not a combined score.

Current analyzer accuracy by supported data-flow languages Scatter plot of all analyzers in DataFlowBench v0.9.2. The horizontal axis is the documented number of languages in which each analyzer performs data-flow analysis. The vertical axis is decisive-correct assertions divided by every assertion in the benchmark kernels that analyzer covers, including non-answers in the denominator. 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0 2 4 6 8 10 12 14 Languages with documented data-flow support Accuracy within covered kernels Bifrost CodeQL Joern Semgrep CE OpenTaint Infer FlowDroid Pysa
  • generalist
  • specialist
Exact figures
AnalyzerScopeSupported languagesBenchmark kernelsAccuracyCorrectWrongIncomplete
Bifrost generalist 13C, C++, C#, Go, Java, JavaScript, Kotlin, PHP, Python, Ruby, Rust, Scala, TypeScript 13/14 (92.9%) 527/886 (59.5%) 5271358
CodeQL generalist 12C, C++, C#, Go, Java, JavaScript, Kotlin, Python, Ruby, Rust, Swift, TypeScript 12/14 (85.7%) 594/812 (73.2%) 59414672
Joern generalist 11C, C++, C#, Go, Java, JavaScript, Kotlin, PHP, Python, Ruby, Swift 7/14 (50.0%) 274/478 (57.3%) 27476128
Semgrep CE generalist 11C, C++, Go, Java, JavaScript, Kotlin, PHP, Python, Ruby, Rust, TypeScript 11/14 (78.6%) 132/746 (17.7%) 13222592
OpenTaint specialist 2Java, Kotlin 2/14 (14.3%) 121/140 (86.4%) 121190
Infer specialist 3C, C++, Java 3/14 (21.4%) 157/194 (80.9%) 157370
FlowDroid specialist 2Java, Kotlin 2/14 (14.3%) 122/140 (87.1%) 122180
Pysa specialist 1Python 1/14 (7.1%) 56/70 (80.0%) 56140

Modeling matrix — is the model surface load-bearing?

A separate population from the kernels above, on the same benchmark-controlled profile. Twelve preregistered templates in six balanced categories — declared sources and sinks, declared propagators, declared sanitizers, opaque summaries, framework entry points, persistence boundaries — ask whether each tool's own model-declaration surface can express a category and be made to carry the flow. A category a tool cannot express is unsupported, decided before the tool is invoked. The scored partition differs per adapter, so the denominators differ by construction and are never pooled or ranked. Read the scored column first: it is the load-bearing number, and the ratio inside it is only meaningful against that tool's own partition.

Bifrost 0.15.1

java

7/7 decided correctly · 1 inconclusive 8 scored · 16 unsupported · 0 runner errors of 24

javascript

4/4 decided correctly · 4 inconclusive 8 scored · 16 unsupported · 0 runner errors of 24

python

7/7 decided correctly · 1 inconclusive 8 scored · 16 unsupported · 0 runner errors of 24

CodeQL 2.27.2

java

24/24 decided correctly 24 scored · 0 unsupported · 0 runner errors of 24

javascript

24/24 decided correctly 24 scored · 0 unsupported · 0 runner errors of 24

python

24/24 decided correctly 24 scored · 0 unsupported · 0 runner errors of 24

swift

inconclusive — no decisive answer · 24 inconclusive 24 scored · 0 unsupported · 0 runner errors of 24

Joern 4.0.660

java

14/16 decided correctly 16 scored · 8 unsupported · 0 runner errors of 24

javascript

14/16 decided correctly 16 scored · 8 unsupported · 0 runner errors of 24

python

14/16 decided correctly 16 scored · 8 unsupported · 0 runner errors of 24

swift

inconclusive — no decisive answer · 24 inconclusive 24 scored · 0 unsupported · 0 runner errors of 24

Semgrep CE 1.181.0

java

10/10 decided correctly 10 scored · 14 unsupported · 0 runner errors of 24

javascript

10/10 decided correctly 10 scored · 14 unsupported · 0 runner errors of 24

python

10/10 decided correctly 10 scored · 14 unsupported · 0 runner errors of 24

OpenTaint v0.4.6

java

12/12 decided correctly 12 scored · 12 unsupported · 0 runner errors of 24

Infer v1.3.0

java

10/10 decided correctly 10 scored · 14 unsupported · 0 runner errors of 24

FlowDroid 2.15.1

java

14/14 decided correctly 14 scored · 10 unsupported · 0 runner errors of 24

Pysa 0.10.0

python

20/20 decided correctly 20 scored · 4 unsupported · 0 runner errors of 24

  • correct
  • wrong
  • incomplete — inconclusive or runner-error; each count is shown per row
  • declined — outside this tool's model surface, decided before it runs

Read the scored partition, not the ratio: across the 18 language tiers it runs from 8 to 24 of the 24 assertions in the tier, so no two of these bars are the same exam. The per-tier counts behind every bar are on the analyzers page, and the template-by-template model-* outcomes on the semantic templates page.

Tool-native probes — what ships and decides on its own

A third population, under the tool-native model profile: six templates run with nothing supplied by DataFlowBench, against whatever ruleset, semantics, or policy pack the product ships. This measures product coverage, not engine accuracy. Tool-native results are never pooled with the benchmark-controlled kernels or the modeling matrix, and never compared number-to-number with them. A tier that is entirely unsupported is a declared decline — the tool ships no source or sink catalog for it — and is never counted as a wrong answer. A runner-error remains a separate failed execution, not a decline or negative finding. Those runs still witness the identity of the binary and ruleset that produced them.

Bifrost 0.15.1

java

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

javascript

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

python

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

CodeQL 2.27.2

java

12 runner errors — no answer 0 scored · 0 unsupported · 12 runner errors of 12

javascript

12 runner errors — no answer 0 scored · 0 unsupported · 12 runner errors of 12

python

12 runner errors — no answer 0 scored · 0 unsupported · 12 runner errors of 12

swift

inconclusive — no decisive answer 12 scored · 0 unsupported · 0 runner errors of 12

Joern 4.0.660

java

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

javascript

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

python

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

swift

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

Semgrep CE 1.181.0

java

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

javascript

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

python

8/12 decided correctly · 4 FP 12 scored · 0 unsupported · 0 runner errors of 12

OpenTaint v0.4.6

java

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

Infer v1.3.0

java

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

FlowDroid 2.15.1

java

declines the tier — declared coverage 0 scored · 12 unsupported · 0 runner errors of 12

Pysa 0.10.0

python

6/12 decided correctly · 6 FN 12 scored · 0 unsupported · 0 runner errors of 12

  • correct
  • wrong
  • incomplete — inconclusive or runner-error; each count is shown per row
  • declined — the product ships no catalog for this tier

12 of the 18 tiers are entirely unsupported — declared declines, never zero scores. 2 tiers produce at least one decisive answer (24 assertions across Pysa and Semgrep CE); 4 other tiers have no decisive answer. Runner errors and inconclusive outcomes remain separately counted on each row. Per-tier coverage is on the analyzers page, the native-* template outcomes on the semantic templates page, and every case — including the runs that decide nothing — on the case evidence page.

Current timing evidence

Descriptive roll-up of retained invocation records. No timing is certified. 21 of 84 planned cold groups have retained reports; 18 groups contain timed invocations; 1,085 of 1,170 retained report rows have a wall-time observation. The 85 unsupported rows have no timed invocation; 338 timed rows have inconclusive outcomes. A recorded duration does not certify successful analysis. This is the 13 November 2026 capture. Rows follow the frozen v0.9.2 analyzer order and pins.

Each analyzer's distribution is computed from its raw per-case observations. It combines mixed languages, tiers, populations and profiles, so it is not a matched comparison, performance ranking or certification. The six analyzers without retained cold reports are shown with their recorded runtime blockers.

8 analyzers · 21 / 84 groups captured · 1,085 / 1,170 report rows timed

Cold whole-invocation wall time, including startup. Dot: median. Tick: 90th percentile. Line: observed minimum to maximum. Logarithmic time scale, shared by every row and filter.
  • Bifrost v0.15.1 At capture: Pinned Scala Maven materialization must be restored and verified.
    961 / 1,046 timed · 19 / 20 groups
  • CodeQL CLI 2.27.2 · frozen packs At capture: Exact v0.9.2 runtime closure is missing from its bound paths; restoration and verification are pending.
    0 / 18 groups captured
  • Joern 4.0.660 At capture: Exact v0.9.2 runtime closure is missing from its bound paths; restoration and verification are pending.
    0 / 13 groups captured
  • Semgrep CE 1.181.0 · rules 17e31e11… At capture: Exact v0.9.2 runtime closure is missing from its bound paths; restoration and verification are pending.
    0 / 17 groups captured
  • OpenTaint 0.4.6 · analyzer 2026.09.04.c51dc3e At capture: Pinned /usr/bin/java or javac digest differs on this OS.
    0 / 4 groups captured
  • Infer v1.3.0 At capture: Pinned /usr/bin/java or javac digest differs on this OS.
    124 / 124 timed · 2 / 5 groups
  • FlowDroid 2.15.1 At capture: Pinned /usr/bin/java or javac digest differs on this OS.
    0 / 4 groups captured
  • Pysa / pyre-check 0.10.0 · Pyrefly 1.4.0 At capture: Exact v0.9.2 runtime closure is missing from its bound paths; restoration and verification are pending.
    0 / 3 groups captured

These values summarize different case invocations, not repeated trials or runtime variance. The Bifrost C cold record retained a passing initial quiet-gate flag; later cold records retained false flags because no renewed per-group quiet checks were made. Contention was observed in Bifrost C and Java cold records, and Bifrost Python overhead only. No CPU observation failures were recorded (0). A24 inter-adapter settling was omitted for the overhead measurements. Process-descendant containment remains unproven, and every recorded certification flag remains false.

Separate invocation-overhead estimates: Bifrost / Python 562 ms–805 ms (contention observed); Infer / C 245 ms–260 ms (no contention observed). Three repeats each; ranges are upper-bound estimates including analysis of no-flow fixtures, not warm marginal cost. Nothing is subtracted from the cold chart.

Full timing tables, qualification and downloadable evidence

Timing values for all 21 captured slices

Rounded for readability. This table always shows the retained per-slice values, regardless of chart filters. A dash means no timing, never zero; full precision is retained in the downloadable evidence.

Uncertified cold-case wall-clock observations
SliceTimed / report rowsMinimumMedian90th percentileMaximum
Bifrost · C kernel58 / 58559 ms598 ms631 ms1.16 s
Bifrost · C++ kernel68 / 68591 ms609 ms634 ms1.1 s
Bifrost · C# kernel70 / 70593 ms613 ms635 ms1 s
Bifrost · Go kernel70 / 70697 ms725 ms758 ms1.18 s
Bifrost · Java kernel70 / 701.35 s1.38 s1.41 s8.36 s
Bifrost · Java modeling8 / 241.38 s1.4 s3.45 s8.14 s
Bifrost · Java native0 / 12————
Bifrost · JavaScript kernel70 / 70596 ms610 ms631 ms980 ms
Bifrost · JavaScript modeling8 / 24591 ms612 ms734 ms987 ms
Bifrost · JavaScript native0 / 12————
Bifrost · Kotlin kernel70 / 701.39 s1.46 s1.57 s8.16 s
Bifrost · PHP kernel70 / 70595 ms611 ms634 ms1.01 s
Bifrost · Python kernel70 / 70591 ms605 ms625 ms967 ms
Bifrost · Python modeling8 / 24593 ms608 ms737 ms995 ms
Bifrost · Python native0 / 12————
Bifrost · Ruby kernel70 / 70588 ms603 ms621 ms976 ms
Bifrost · Rust kernel64 / 64655 ms679 ms723 ms1.09 s
Bifrost · mixed languages smoke117 / 118603 ms629 ms1.41 s7.75 s
Bifrost · TypeScript kernel70 / 70598 ms613 ms630 ms1.01 s
Infer · C kernel56 / 56254 ms263 ms281 ms964 ms
Infer · C++ kernel68 / 68258 ms268 ms1.63 s1.82 s
Progress over time

How the benchmark and its analyzers evolved

Generalists

One line per analyzer across every freeze. The vertical axis counts decisive-correct assertions; the grey step is the full benchmark core, which grew from 32 assertions in v0.1.0 to 952 in v0.9.2. Nothing is pooled with other result profiles or turned into a combined score.

Decisive-correct assertions per analyzer across DataFlowBench snapshots A stepped grey line shows each snapshot's total benchmark-controlled core population, growing from 32 assertions in v0.1.0 to 952 in v0.9.2. Beneath it, one line per analyzer plots that analyzer's decisive-correct count, with a faint stub reaching that analyzer's own covered population. Every marker also carries its exact figures in a tooltip, and the toggle above this chart has a Data table view that lists every value as text. 020040060080010003296316738738738738738738886952952v0.1.01kernelv0.2.03kernelsv0.3.010kernelsv0.4.013kernelsv0.5.013kernelsv0.6.013kernelsv0.6.113kernelsv0.7.013kernelsv0.7.113kernelsv0.8.013kernelsv0.9.014kernelsv0.9.214kernels Assertions
Decisive-correct share of each analyzer's own covered population across DataFlowBench snapshots One line per analyzer on a nought to one hundred percent axis. Each point is that analyzer's decisive-correct count divided by the assertions in the kernels it covers, with inconclusive and unsupported outcomes left in the denominator. Marker size grows with the share of the snapshot's kernels the analyzer covers, and every point is annotated with that kernel count. Every marker also carries its exact figures in a tooltip, and the toggle above this chart has a Data table view that lists every value as text, including the denominator each percentage is over. 0% 20% 40% 60% 80% 100% 1/13/310/1013/1313/1313/1313/1313/1313/1313/1313/1413/141/13/310/1011/1311/1311/1311/1311/1311/1311/1312/1412/146/136/136/136/136/136/136/137/147/1411/1311/1311/1311/1311/1311/1311/1311/1411/14v0.1.01kernelv0.2.03kernelsv0.3.010kernelsv0.4.013kernelsv0.5.013kernelsv0.6.013kernelsv0.6.113kernelsv0.7.013kernelsv0.7.113kernelsv0.8.013kernelsv0.9.014kernelsv0.9.214kernels Share of its covered population
  • Bifrost · decisive-correct
  • CodeQL · decisive-correct
  • Joern · decisive-correct
  • Semgrep CE · decisive-correct
  • benchmark core population (the denominator)
  • remainder of that analyzer's own covered population
  • Bifrost · decisive-correct ÷ its covered population
  • CodeQL · decisive-correct ÷ its covered population
  • Joern · decisive-correct ÷ its covered population
  • Semgrep CE · decisive-correct ÷ its covered population
  • marker size and the k/n label under it: kernels covered of the snapshot's kernels

Marker: decisive-correct. Cap or stub: that analyzer's covered population. Grey step: the full core population. The covered-kernel view keeps non-answers in its denominator and shows coverage as k/n; missing runs are absent, never zero. Exact figures are in the table.

SnapshotAnalyzerDecisive-correctDecidedCoverage outcomesIts covered populationCorrect ÷ its covered populationBenchmark core populationSecondary: correct ÷ decided
v0.1.0Bifrost17221032(1 of 1 kernel)53.1%(17 of 32 covered, 1 of 1 kernel)3277.3% (17 of 22 decided)
v0.1.0CodeQL2732032(1 of 1 kernel)84.4%(27 of 32 covered, 1 of 1 kernel)3284.4% (27 of 32 decided)
v0.2.0Bifrost52682896(3 of 3 kernels)54.2%(52 of 96 covered, 3 of 3 kernels)9676.5% (52 of 68 decided)
v0.2.0CodeQL8496096(3 of 3 kernels)87.5%(84 of 96 covered, 3 of 3 kernels)9687.5% (84 of 96 decided)
v0.3.0Bifrost163166150316(10 of 10 kernels)51.6%(163 of 316 covered, 10 of 10 kernels)31698.2% (163 of 166 decided)
v0.3.0CodeQL2763160316(10 of 10 kernels)87.3%(276 of 316 covered, 10 of 10 kernels)31687.3% (276 of 316 decided)
v0.4.0Bifrost222227511738(13 of 13 kernels)30.1%(222 of 738 covered, 13 of 13 kernels)73897.8% (222 of 227 decided)
v0.4.0CodeQL5066220622(11 of 13 kernels)81.4%(506 of 622 covered, 11 of 13 kernels)73881.4% (506 of 622 decided)
v0.4.0Joern2703440344(6 of 13 kernels)78.5%(270 of 344 covered, 6 of 13 kernels)73878.5% (270 of 344 decided)
v0.4.0Semgrep CE132154468622(11 of 13 kernels)21.2%(132 of 622 covered, 11 of 13 kernels)73885.7% (132 of 154 decided)
v0.5.0Bifrost435440298738(13 of 13 kernels)58.9%(435 of 738 covered, 13 of 13 kernels)73898.9% (435 of 440 decided)
v0.5.0CodeQL5066220622(11 of 13 kernels)81.4%(506 of 622 covered, 11 of 13 kernels)73881.4% (506 of 622 decided)
v0.5.0Joern2703440344(6 of 13 kernels)78.5%(270 of 344 covered, 6 of 13 kernels)73878.5% (270 of 344 decided)
v0.5.0Semgrep CE132154468622(11 of 13 kernels)21.2%(132 of 622 covered, 11 of 13 kernels)73885.7% (132 of 154 decided)
v0.6.0Bifrost435440298738(13 of 13 kernels)58.9%(435 of 738 covered, 13 of 13 kernels)73898.9% (435 of 440 decided)
v0.6.0CodeQL5066220622(11 of 13 kernels)81.4%(506 of 622 covered, 11 of 13 kernels)73881.4% (506 of 622 decided)
v0.6.0Joern2703440344(6 of 13 kernels)78.5%(270 of 344 covered, 6 of 13 kernels)73878.5% (270 of 344 decided)
v0.6.0Semgrep CE132154468622(11 of 13 kernels)21.2%(132 of 622 covered, 11 of 13 kernels)73885.7% (132 of 154 decided)
v0.6.1Bifrost446446292738(13 of 13 kernels)60.4%(446 of 738 covered, 13 of 13 kernels)738100.0% (446 of 446 decided)
v0.6.1CodeQL5066220622(11 of 13 kernels)81.4%(506 of 622 covered, 11 of 13 kernels)73881.4% (506 of 622 decided)
v0.6.1Joern2703440344(6 of 13 kernels)78.5%(270 of 344 covered, 6 of 13 kernels)73878.5% (270 of 344 decided)
v0.6.1Semgrep CE132154468622(11 of 13 kernels)21.2%(132 of 622 covered, 11 of 13 kernels)73885.7% (132 of 154 decided)
v0.7.0Bifrost450450288738(13 of 13 kernels)61.0%(450 of 738 covered, 13 of 13 kernels)738100.0% (450 of 450 decided)
v0.7.0CodeQL5016175622(11 of 13 kernels)80.5%(501 of 622 covered, 11 of 13 kernels)73881.2% (501 of 617 decided)
v0.7.0Joern2703440344(6 of 13 kernels)78.5%(270 of 344 covered, 6 of 13 kernels)73878.5% (270 of 344 decided)
v0.7.0Semgrep CE132154468622(11 of 13 kernels)21.2%(132 of 622 covered, 11 of 13 kernels)73885.7% (132 of 154 decided)
v0.7.1Bifrost452452286738(13 of 13 kernels)61.2%(452 of 738 covered, 13 of 13 kernels)738100.0% (452 of 452 decided)
v0.7.1CodeQL5016175622(11 of 13 kernels)80.5%(501 of 622 covered, 11 of 13 kernels)73881.2% (501 of 617 decided)
v0.7.1Joern2703440344(6 of 13 kernels)78.5%(270 of 344 covered, 6 of 13 kernels)73878.5% (270 of 344 decided)
v0.7.1Semgrep CE132154468622(11 of 13 kernels)21.2%(132 of 622 covered, 11 of 13 kernels)73885.7% (132 of 154 decided)
v0.8.0Bifrost494494392886(13 of 13 kernels)55.8%(494 of 886 covered, 13 of 13 kernels)886100.0% (494 of 494 decided)
v0.8.0CodeQL499619127746(11 of 13 kernels)66.9%(499 of 746 covered, 11 of 13 kernels)88680.6% (499 of 619 decided)
v0.8.0Joern3234120412(6 of 13 kernels)78.4%(323 of 412 covered, 6 of 13 kernels)88678.4% (323 of 412 decided)
v0.8.0Semgrep CE132154592746(11 of 13 kernels)17.7%(132 of 746 covered, 11 of 13 kernels)88685.7% (132 of 154 decided)
v0.9.0Bifrost524524362886(13 of 14 kernels)59.1%(524 of 886 covered, 13 of 14 kernels)952100.0% (524 of 524 decided)
v0.9.0CodeQL59674171812(12 of 14 kernels)73.4%(596 of 812 covered, 12 of 14 kernels)95280.4% (596 of 741 decided)
v0.9.0Joern32341266478(7 of 14 kernels)67.6%(323 of 478 covered, 7 of 14 kernels)95278.4% (323 of 412 decided)
v0.9.0Semgrep CE132154592746(11 of 14 kernels)17.7%(132 of 746 covered, 11 of 14 kernels)95285.7% (132 of 154 decided)
v0.9.2Bifrost527528358886(13 of 14 kernels)59.5%(527 of 886 covered, 13 of 14 kernels)95299.8% (527 of 528 decided)
v0.9.2CodeQL59474072812(12 of 14 kernels)73.2%(594 of 812 covered, 12 of 14 kernels)95280.3% (594 of 740 decided)
v0.9.2Joern274350128478(7 of 14 kernels)57.3%(274 of 478 covered, 7 of 14 kernels)95278.3% (274 of 350 decided)
v0.9.2Semgrep CE132154592746(11 of 14 kernels)17.7%(132 of 746 covered, 11 of 14 kernels)95285.7% (132 of 154 decided)

The exact values behind both chart views, one row per analyzer and snapshot.

Specialists

Analyzers deliberately focused on one ecosystem or a small related family, separated because breadth and specialization are not directly comparable. Uses the same percentage axis as the generalists' covered-kernel view.

Decisive-correct share of each analyzer's own covered population across DataFlowBench snapshots One line per analyzer on a nought to one hundred percent axis. Each point is that analyzer's decisive-correct count divided by the assertions in the kernels it covers, with inconclusive and unsupported outcomes left in the denominator. Marker size grows with the share of the snapshot's kernels the analyzer covers, and every point is annotated with that kernel count. Every marker also carries its exact figures in a tooltip, and the toggle above this chart has a Data table view that lists every value as text, including the denominator each percentage is over. 0% 20% 40% 60% 80% 100% 2/132/132/132/132/132/142/143/133/133/133/133/133/143/142/132/132/132/132/132/142/141/131/131/131/131/131/141/14v0.6.013kernelsv0.6.113kernelsv0.7.013kernelsv0.7.113kernelsv0.8.013kernelsv0.9.014kernelsv0.9.214kernels Share of its covered population
  • OpenTaint · decisive-correct ÷ its covered population
  • Infer · decisive-correct ÷ its covered population
  • FlowDroid · decisive-correct ÷ its covered population
  • Pysa · decisive-correct ÷ its covered population
  • marker size and the k/n label under it: kernels covered of the snapshot's kernels

Same scales and encoding as the generalists. The covered-kernel view keeps specialist scope visible as k/n; exact figures are in the table.

SnapshotAnalyzerDecisive-correctDecidedCoverage outcomesIts covered populationCorrect ÷ its covered populationBenchmark core populationSecondary: correct ÷ decided
v0.6.0OpenTaint991160116(2 of 13 kernels)85.3%(99 of 116 covered, 2 of 13 kernels)73885.3% (99 of 116 decided)
v0.6.0Infer1401620162(3 of 13 kernels)86.4%(140 of 162 covered, 3 of 13 kernels)73886.4% (140 of 162 decided)
v0.6.0FlowDroid981160116(2 of 13 kernels)84.5%(98 of 116 covered, 2 of 13 kernels)73884.5% (98 of 116 decided)
v0.6.0Pysa4758058(1 of 13 kernels)81.0%(47 of 58 covered, 1 of 13 kernels)73881.0% (47 of 58 decided)
v0.6.1OpenTaint991160116(2 of 13 kernels)85.3%(99 of 116 covered, 2 of 13 kernels)73885.3% (99 of 116 decided)
v0.6.1Infer1401620162(3 of 13 kernels)86.4%(140 of 162 covered, 3 of 13 kernels)73886.4% (140 of 162 decided)
v0.6.1FlowDroid981160116(2 of 13 kernels)84.5%(98 of 116 covered, 2 of 13 kernels)73884.5% (98 of 116 decided)
v0.6.1Pysa4758058(1 of 13 kernels)81.0%(47 of 58 covered, 1 of 13 kernels)73881.0% (47 of 58 decided)
v0.7.0OpenTaint991160116(2 of 13 kernels)85.3%(99 of 116 covered, 2 of 13 kernels)73885.3% (99 of 116 decided)
v0.7.0Infer1401620162(3 of 13 kernels)86.4%(140 of 162 covered, 3 of 13 kernels)73886.4% (140 of 162 decided)
v0.7.0FlowDroid981160116(2 of 13 kernels)84.5%(98 of 116 covered, 2 of 13 kernels)73884.5% (98 of 116 decided)
v0.7.0Pysa4758058(1 of 13 kernels)81.0%(47 of 58 covered, 1 of 13 kernels)73881.0% (47 of 58 decided)
v0.7.1OpenTaint1011160116(2 of 13 kernels)87.1%(101 of 116 covered, 2 of 13 kernels)73887.1% (101 of 116 decided)
v0.7.1Infer1401620162(3 of 13 kernels)86.4%(140 of 162 covered, 3 of 13 kernels)73886.4% (140 of 162 decided)
v0.7.1FlowDroid981160116(2 of 13 kernels)84.5%(98 of 116 covered, 2 of 13 kernels)73884.5% (98 of 116 decided)
v0.7.1Pysa4758058(1 of 13 kernels)81.0%(47 of 58 covered, 1 of 13 kernels)73881.0% (47 of 58 decided)
v0.8.0OpenTaint1211400140(2 of 13 kernels)86.4%(121 of 140 covered, 2 of 13 kernels)88686.4% (121 of 140 decided)
v0.8.0Infer1571940194(3 of 13 kernels)80.9%(157 of 194 covered, 3 of 13 kernels)88680.9% (157 of 194 decided)
v0.8.0FlowDroid1221400140(2 of 13 kernels)87.1%(122 of 140 covered, 2 of 13 kernels)88687.1% (122 of 140 decided)
v0.8.0Pysa5670070(1 of 13 kernels)80.0%(56 of 70 covered, 1 of 13 kernels)88680.0% (56 of 70 decided)
v0.9.0OpenTaint1211400140(2 of 14 kernels)86.4%(121 of 140 covered, 2 of 14 kernels)95286.4% (121 of 140 decided)
v0.9.0Infer1571940194(3 of 14 kernels)80.9%(157 of 194 covered, 3 of 14 kernels)95280.9% (157 of 194 decided)
v0.9.0FlowDroid1221400140(2 of 14 kernels)87.1%(122 of 140 covered, 2 of 14 kernels)95287.1% (122 of 140 decided)
v0.9.0Pysa5670070(1 of 14 kernels)80.0%(56 of 70 covered, 1 of 14 kernels)95280.0% (56 of 70 decided)
v0.9.2OpenTaint1211400140(2 of 14 kernels)86.4%(121 of 140 covered, 2 of 14 kernels)95286.4% (121 of 140 decided)
v0.9.2Infer1571940194(3 of 14 kernels)80.9%(157 of 194 covered, 3 of 14 kernels)95280.9% (157 of 194 decided)
v0.9.2FlowDroid1221400140(2 of 14 kernels)87.1%(122 of 140 covered, 2 of 14 kernels)95287.1% (122 of 140 decided)
v0.9.2Pysa5670070(1 of 14 kernels)80.0%(56 of 70 covered, 1 of 14 kernels)95280.0% (56 of 70 decided)

The exact values behind both chart views, one row per analyzer and snapshot.

Kernels at a glance

One bar per analyzer, one panel per kernel. Each bar is that analyzer's own core population, split into decisive-correct, decisive-wrong, incomplete, and declined. The incomplete segment combines inconclusive proof and runner-error executions, which the panel labels count separately; declined means unsupported only. The segments ensure the remainder of a bar is never read as a wrong answer. Panels are separate populations with separate denominators: compare bars inside a panel, never across panels, and never against the modeling or tool-native sections above. Every kernel here is on the benchmark-controlled model profile.

c 56 assertions

Bifrost

42/56

CodeQL

47/56

Infer

47/56

Semgrep CE

12/56

cpp 68 assertions

Bifrost

30/68

CodeQL

50/68

Infer

53/68

Semgrep CE

12/68

csharp 70 assertions

Bifrost

43/70

CodeQL

55/70

go 70 assertions

Bifrost

44/70

CodeQL

54/70

Semgrep CE

12/70

java 70 assertions

Bifrost

46/70

CodeQL

57/70

FlowDroid

61/70

Infer

57/70

Joern

57/70

OpenTaint

61/70

Semgrep CE

12/70

javascript 70 assertions

Bifrost

48/70

CodeQL

57/70

Joern

55/70

Semgrep CE

12/70

kotlin 70 assertions

Bifrost

32/70

CodeQL

55/70

FlowDroid

61/70

OpenTaint

60/70

Semgrep CE

12/70

php 70 assertions

Bifrost

34/70

Joern

58/70

Semgrep CE

12/70

python 70 assertions

Bifrost

42/70

CodeQL

56/70

Joern

58/70

Pysa

56/70

Semgrep CE

12/70

ruby 70 assertions

Bifrost

36/70

CodeQL

57/70

Joern

46/70

Semgrep CE

12/70

rust 62 assertions

Bifrost

42/62

CodeQL

49/62

Joern

0/62

Semgrep CE

12/62

scala 70 assertions

Bifrost

42/70

swift 66 assertions

CodeQL

0/66

Joern

0/66

typescript 70 assertions

Bifrost

46/70

CodeQL

57/70

Semgrep CE

12/70

  • decisive-correct
  • decisive-wrong
  • incomplete — inconclusive proof or runner-error execution; separately labeled and neither a wrong answer
  • declined — unsupported by declared capability, decided before the analyzer runs

None of the 51 analyzer rows published across these 14 kernels answers its whole core correctly, and 18 of them answer every assertion in their kernel definitively — with no inconclusive results, unsupported outcomes, or runner errors. The template-by-template outcome for every one of these rows — each balanced positive/negative pair, per analyzer — is on the semantic templates page, with per-case classifications and retained raw evidence on the case evidence page.

Direct-flow breadth across languages

The breadth baseline is one balanced direct-propagation pair per language, on Bifrost's all-language smoke run: a separate, much smaller population from the kernels above, and never pooled with them. Each chip is one language, with a mark for the positive and the negative assertion.

  • c
  • cpp
  • csharp
  • go
  • java
  • javascript
  • kotlin
  • php
  • python
  • ruby
  • rust
  • scala
  • typescript

Bifrost answers both assertions correctly in 13 of 13 languages. The exact reached / not-reached outcome for each of those assertions is on the semantic templates page, under the smoke scorecard's direct-propagation rows.

Why these numbers are trustworthy

Every count on this page is generated from the freeze manifest (750e3b8c1275…), which digest-binds the benchmark revision, every case and fixture, every analyzer's identity, their normalized reports, and one retained raw artifact per result. CI regenerates the numbers from the manifest and fails on any drift; hand-authored prose cannot override a generated count. See reproduction to verify locally, or dive into the full snapshot — analyzers, languages, templates, and per-case evidence.