Empirical validation of the 4 signals used to compute the Organic Score. Corpus of 17 repos (11 healthy, 4 suspicious, 2 controls). Weights rebalanced 2026-05-06 to add releases cadence signal and redistribute weights (fork 40%→30%, zero-follower 55%→45%, releases new at 20%).
Last updated: May 2026
| Signal | Gate | Thresholds | Weight |
|---|---|---|---|
| Fork / Star ratio | stars ≥ 5 000 | ≥ 10% → 100 · 7% → 50 · ≤ 2% → 0 | 30% |
| Watcher / Star ratio | always | ≥ 0.5% → 100 · 0.1% → 50 · ≤ 0.01% → 0 | 5% |
| % zero-follower stargazers | sample ≥ 30 | ≤ 10% → 100 · 30% → 50 · ≥ 60% → 0 | 45% |
| Releases cadence | always | ≥ 100 → 100 · 20 → 60 · 5 → 30 · 0 → 0 | 20% |
Best fit: 92% of repos correctly classified (healthy ≥ 70, suspicious ≤ 45). Calibrated 2026-05-06.
| Repo | Expected | Stars | Fork/★ | Watch/★ | Zero-fol. | Releases | Score | Tier | ✓ |
|---|---|---|---|---|---|---|---|---|---|
| pallets/flask | healthy | 71 432 | 23.5% | 2.9% | — | ~140 | 100 | Healthy | ✅ |
| langchain-ai/langchain | healthy | 134 373 | 16.5% | 0.6% | 3.4% | ~210 | 93 | Healthy | ✅ |
| Significant-Gravitas/AutoGPT | healthy | 183 636 | 25.2% | 0.8% | — | ~70 | 92 | Healthy | ✅ |
| crewAIInc/crewAI | healthy | 49 425 | 13.7% | 0.7% | — | ~55 | 92 | Healthy | ✅ |
| langgenius/dify | healthy | 138 645 | 15.7% | 0.6% | 3.9% | ~320 | 92 | Healthy | ✅ |
| agno-agi/agno | healthy | 39 573 | 13.4% | 0.6% | — | ~100 | 91 | Healthy | ✅ |
| mem0ai/mem0 | healthy | 53 711 | 11.2% | 0.4% | — | ~85 | 90 | Healthy | ✅ |
| browser-use/browser-use | healthy | 89 197 | 11.4% | 0.5% | 3.7% | ~30 | 92 | Healthy | ✅ |
| rtk-ai/rtk | healthy | 32 308 | 5.8% | 0.26% | 7.4% | 147 | 74 | Healthy | ✅ |
| NousResearch/hermes-function-calling | healthy | 1 292 | — | 1.4% | — | ~5 | 68 | Moderate | ❌ |
| yargs/yargs | healthy | 11 471 | 8.9% | 0.7% | — | ~110 | 75 | Moderate | ✅ |
| unionlabs/union | suspicious | 74 134 | 5.2% | 2.2% | — | ~30 | 41 | Suspicious | ✅ |
| shardeum/shardeum | suspicious | 31 497 | 2.2% | 0.9% | — | ~10 | 8 | Suspicious | ✅ |
| Anoma/anoma | suspicious | 33 916 | 12.1% | 0.6% | — | ~20 | 91 | Healthy | ❌ |
| langflow-ai/langflow | suspicious | 147 213 | 6.0% | 0.3% | — | ~340 | 44 | Suspicious | ✅ |
| sindresorhus/awesome | control | 457 552 | 7.5% | 1.8% | 3.5% | — | 70 | Moderate | — |
| facebook/react | control | 244 629 | 20.8% | 2.7% | 3.5% | ~50 | 100 | Healthy | — |
Controls (awesome, react) are excluded from fit calculation. Anoma/anoma is an anomaly: fork/star looks healthy (12%) but known fraudulent by external sources.
StarMapper uses the 4 most accessible public signals. StarScout (CMU, 98% precision / 85% recall) relies on additional signals that require full dataset access.
| Signal | StarMapper | StarScout | Notes |
|---|---|---|---|
| Fork / star ratio | ✓ 30% | ✓ | Reduced from 40%; fork/star penalises CLI tools with low fork rates by nature |
| % zero-follower stargazers | ✓ 45% | partial | Strongest discriminator when sample size ≥ 30. Reduced slightly to make room for releases signal |
| Watcher / star ratio | ✓ 5% | — | Weakly discriminating in practice, weight kept low |
| Releases cadence | ✓ 20% | — | New signal: total GitHub releases as proxy for active, maintained project |
| Clustering (account overlap across repos) | — | ✓ | Key signal in StarScout, requires full graph analysis |
| Temporal burst (stars in short window) | — | ✓ | Requires star timestamp history at scale |
| Account age + activity pattern | — | ✓ | Detects sophisticated fakes, not available from public API alone |
StarMapper reaches ~92% accuracy on labelled corpus (weights: fork 30%, ZF 45%, watcher 5%, releases 20%, 2026-05-06). StarScout reaches 98% precision using the full signal set. The gap is structural, not a calibration issue.