CmdID

CmdID / Accuracy

Accuracy

How often CmdID reads a real command line or configuration as what it is, per platform. The lines are taken from public corpora: reference examples, tldr-pages and Batfish's test configurations, none of them the files an index is built from. Measured on 2026-11-16, with the engine at commit 6121af974a42: 37,321 lines, 97.3% of them detected.

Scores

Per platform, and per corpus under it. A share is rounded down to a tenth of a per cent. The measures are explained below.

CorpusLinesDetectedExactDescribed
aws7,39899.5%7,363
aws-examplea command line from aws-cli's examples (awscli/examples)7,06999.8%7,05799.1%7,01097.5%8,161 of 8,364 options
tldr-awsan aws line from tldr-pages32993.0%306-91.8%381 of 415 options
az6,58799.7%6,573
az-examplethe first line of an example in an Azure CLI command's help6,39599.9%6,39299.6%6,37091.7%20,534 of 22,392 options
tldr-azan az line from tldr-pages19294.2%181-82.6%324 of 392 options
docker1,00299.6%998
docker-examplea command line from docker's command reference73999.5%736-98.9%687 of 694 options
tldr-dockera docker line from tldr-pages26399.6%262-99.4%181 of 182 options
kubectl60499.8%603
kubectl-examplea command line from the kubectl reference's examples381100.0%381-100.0%457 of 457 options
tldr-kubectla kubectl line from tldr-pages22399.5%222-100.0%190 of 190 options
gcloud9298.9%91
tldr-gclouda gcloud line from tldr-pages9298.9%91-100.0%43 of 43 options
helm1593.3%14
tldr-helma helm line from tldr-pages1593.3%14-100.0%4 of 4 options
terraform105100.0%105
tldr-terraforma terraform line from tldr-pages105100.0%105-89.0%57 of 64 options
vault7100.0%7
tldr-vaulta vault line from tldr-pages7100.0%7-100.0%1 of 1 options
openssl5098.0%49
tldr-opensslan openssl line from tldr-pages5098.0%49-100.0%107 of 107 options
git77598.9%767
tldr-gita git line from tldr-pages77598.9%767-92.5%437 of 472 options
npm283100.0%283
tldr-npman npm line from tldr-pages283100.0%283-92.3%108 of 117 options
pip84100.0%84
tldr-pipa pip line from tldr-pages84100.0%84-100.0%52 of 52 options
ansible42100.0%42
tldr-ansiblean ansible line from tldr-pages (ansible, ansible-playbook, ansible-galaxy ...)42100.0%42-90.0%45 of 50 options
PowerShell9,68699.7%9,659
ps-winserveran example line that calls its page's cmdlet, up to two per page4,38999.9%4,386-96.2%7,539 of 7,832 parameters
ps-officean example line that calls its page's cmdlet, up to two per page2,85699.1%2,832-97.5%4,659 of 4,778 parameters
ps-graphan example line that calls its page's cmdlet, up to two per page2,441100.0%2,441-98.7%3,888 of 3,936 parameters
Junos5,05698.6%4,989
junos-linea set line of a Junos test configuration5,03098.6%4,96496.4%4,84993.0%21,186 of 22,768 configuration lines
junos-curlya Junos test configuration in the { } form, whole2696.1%2592.3%2494.1%415 of 441 configuration lines
Cisco IOS XE1,26779.5%1,008
ios-stanzaa top-level line of a Cisco test configuration, with the lines under it, from a file not naming an OS other than IOS or IOS XE1,08579.0%858-31.5%919 of 2,913 configuration lines
ios-napalma top-level line of a configuration in napalm's IOS tests, with the lines under it7386.3%63-33.3%66 of 198 configuration lines
ios-ansiblea top-level line of a cisco.ios unit-test configuration, with the lines under it10979.8%87-49.2%163 of 331 configuration lines
Before, ios-stanza-allevery Cisco test configuration read, as scored before the split by OS1,08579.0%858
FortiOS31099.6%309
fortios-blocka top-level config ... end block of a FortiOS test configuration31099.6%309-99.2%1,125 of 1,134 configuration lines
PAN-OS2,79699.4%2,780
panos-linea set line of a PAN-OS test configuration2,77499.4%2,75897.6%2,70953.2%6,597 of 12,387 configuration lines
panos-curlya PAN-OS test configuration in the { } form, whole22100.0%2286.3%1970.9%630 of 888 configuration lines
Arista EOS1,16251.2%595
eos-stanzaa top-level line of an Arista EOS test configuration, with the lines under it98553.5%527-28.6%1,168 of 4,076 configuration lines
eos-napalma top-level line of a configuration in napalm's EOS tests, with the lines under it4139.0%16-51.4%73 of 142 configuration lines
eos-ansiblea top-level line of an arista.eos unit-test configuration, with the lines under it13638.2%52-41.3%81 of 196 configuration lines
Cisco IOS XE, whole configurations20697.5%201
ios-filea Cisco test configuration, whole, from a file not naming an OS other than IOS or IOS XE17097.0%165-33.7%1,082 of 3,203 configuration lines
ios-napalm-filea configuration in napalm's IOS tests, whole17100.0%17-46.4%304 of 655 configuration lines
ios-ansible-filea cisco.ios unit-test configuration, whole19100.0%19-46.7%168 of 359 configuration lines
Arista EOS, whole configurations17685.7%151
eos-filean Arista EOS test configuration, whole12092.5%111-31.0%1,502 of 4,841 configuration lines
eos-napalm-filea configuration in napalm's EOS tests, whole1782.3%14-48.5%103 of 212 configuration lines
eos-ansible-filean arista.eos unit-test configuration, whole3966.6%26-39.3%141 of 358 configuration lines
All platforms37,32197.3%36,319

A row headed Before is the platform's score as it was counted before a change to its corpus, kept beside the new one for one release.

What each device set was read as

Per device set, how many of its lines each device reader claimed (the set's own reader's count in bold), how many were read as no device - a program, a cmdlet or something else - and how many nothing was found for. The Cisco set is split by what each file says its OS is: its name (ios-xe-...), a RANCID header naming a type, an ASA or PIX version line, or a version line of a release IOS XE has and IOS never had; a file that says none of these is IOS or IOS XE, not said which, and no OS is told from how a stanza looks. Batfish keeps its ASA, IOS XR and NX-OS test configurations in folders of their own, which are not read.

SetLinesFortiOSCisco IOS XE CLIJunos CLIPAN-OS CLIArista EOS CLIa programnothing found
junos-line5,030004,96400660
junos-curly2600250001
ios-stanzafiles naming IOS XE38025000130
ios-stanzaIOS or IOS XE, the file not saying which1,04708330072025
fortios-block310309000010
panos-line2,7740002,7580160
panos-curly2200022000
eos-stanza98502770052716021
ios-napalm7306300082
ios-ansible109087005710
eos-napalm41021001622
eos-ansible136069005296
ios-filefiles naming IOS XE70700000
ios-fileIOS or IOS XE, the file not saying which163015800050
ios-napalm-file1701700000
ios-ansible-file1901900000
eos-file120050011140
eos-napalm-file1703001400
eos-ansible-file39010002621

Left out of eos-napalm-file: 4 files (show_running_config___include_service_routing_protocols_model_multi_agent.text, show_running_config___include_service_routing_protocols_model_multi_agent.text, show_running_config___include_service_routing_protocols_model_multi_agent.text, show_running_config___section_router_bgp.text): no line of configuration.

Left out of ios-file: 1 file (ios-unicode): NUL bytes and no byte-order mark: an encoding the file does not say.

Left out of ios-stanza: 1 file (ios-unicode): NUL bytes and no byte-order mark: an encoding the file does not say.

Left out of junos-line, junos-curly: 1 file (nested-config-with-multiline-comment): no set line and no { } block.

Everyday shell lines

30,238 everyday shell lines from tldr-pages/tldr (its Linux, macOS, Windows and common pages, without those of the programs scored above: aws, az, docker, kubectl, gcloud, helm, terraform, vault, openssl, git, npm, pip and ansible). None of them is a network device's configuration; each one a device reader claims is a false positive.

Read asLines
NetScaler ADC CLI0
FortiOS0
Cisco IOS XE CLI0
Junos CLI0
PAN-OS CLI0
RouterOS CLI0
Arista EOS CLI0

What is measured, and how

tools/eval/run.sh in CmdID's repository fetches each corpus at the commit listed below, takes its lines (tools/eval/corpus.py), runs the identifier on each, as the site's engine does, and counts (tools/eval/score.py). The same run writes the file this page is built from.

Detected
The share of lines the identifier read as what they are. What that means differs by platform:
  • aws: a service and an operation found
  • az: a command found, not a group
  • docker: a command found, not only the group above an unknown word
  • kubectl: a command found, not only the group above an unknown word
  • gcloud: a command found, not only the group above an unknown word
  • helm: a command found, not only the group above an unknown word
  • terraform: a command found, not only the group above an unknown word
  • vault: a command found, not only the group above an unknown word
  • openssl: a command found
  • git: a command found (git is read from tldr-pages)
  • npm: a command found, not only the group above an unknown word
  • pip: a command found, not only the group above an unknown word
  • ansible: a command found, or a program that is its own command
  • PowerShell: the page's cmdlet described
  • Junos: read as Junos
  • Cisco IOS XE: read as IOS XE
  • FortiOS: read as FortiOS
  • PAN-OS: read as PAN-OS
  • Arista EOS: read as Arista EOS
  • Cisco IOS XE, whole configurations: read as IOS XE, each file pasted whole
  • Arista EOS, whole configurations: read as Arista EOS, each file pasted whole
Exact
Where the source gives the answer, the share of lines read as that answer:
  • aws-example: the operation the example's file is named for
  • az-example: the command whose help it is in
  • junos-line: every word of the line a place the schema has
  • junos-curly: every word of the configuration a place the schema has
  • panos-line: every word of the line a place the tree has
  • panos-curly: every word of the configuration a place the tree has
Described
Of the options, parameters or configuration lines in the lines detected, the share the identifier gives a description from its reference for. A FortiOS edit line names an object and is not counted.

What the scores do not measure

Corpora

Only counts are shown on this site; no line of any corpus is. Each is read at the commit below, from the folders listed.

CorpusCommitLicenceRead
batfish/batfish57ae1956686b2026-11-12Apache License 2.0projects/batfish/src/test/resources/org/batfish/grammar/juniper, projects/batfish/src/test/resources/org/batfish/grammar/cisco, projects/batfish/src/test/resources/org/batfish/grammar/fortios, projects/batfish/src/test/resources/org/batfish/grammar/palo_alto, projects/batfish/src/test/resources/org/batfish/vendor/arista/grammar
tldr-pages/tldr491149904dee2026-11-14CC BY 4.0pages
aws/aws-clif9ee5263749e2026-11-13Apache License 2.0awscli/examples
Azure/azure-clif280f6de0b992026-11-13MIT Licensesrc/azure-cli/azure/cli/command_modules/*/_help.py, src/azure-cli/azure/cli/command_modules/*/**/_help.py
docker/clifdd4b8e5da332026-11-13Apache License 2.0docs/reference/commandline
kubernetes/websitefe0b202984832026-11-13CC BY 4.0content/en/docs/reference/kubectl/generated
MicrosoftDocs/windows-powershell-docs2029b4982aec2026-11-12CC BY 4.0docset/winserver2025-ps
MicrosoftDocs/office-docs-powershell7cfde21b7f662026-11-13CC BY 4.0exchange/exchange-ps/ExchangePowerShell, teams/teams-ps/MicrosoftTeams
MicrosoftDocs/microsoftgraph-docs-powershellee4ec05b3c952026-10-02CC BY 4.0microsoftgraph/graph-powershell-1.0
napalm-automation/napalm820a06b2069e2026-08-12Apache License 2.0test/eos, test/ios
ansible-collections/arista.eos485c00dbb0a82026-11-10GNU GPL 3.0tests/unit/modules/network/eos/fixtures
ansible-collections/cisco.ios9102b81cb31b2026-11-12GNU GPL 3.0tests/unit/modules/network/ios/fixtures

The identifier's own sources and their licences: Sources and licences.