CmdID / az / az aiinfraoptimization
az aiinfraoptimization recommendvmsku
Recommends Azure virtual machine sizes for an AI workload in the specified subscription and azure location. Azure CLI extensions reference
1 example 6 options Its page in the Azure CLI extensions reference
Examples
Recommend virtual machine sizes for an AI inference workload
az aiinfraoptimization recommendvmsku --location eastus2 --model "{hugging-face-model-id:meta-llama/Llama-3.1-70B-Instruct,parameter-count:70553706496,weight-precision:FP16,num-layers:80,num-key-value-heads:8,head-dim:128,max-context-length:131072}" --workload "{max-concurrent-requests-per-replica:1,token-budget:{prompt-tokens:8000,max-output-tokens:1024}}" --deployment "{replica-count:1,procurement-option:OnDemand}" --optimization "{ranking-preference:Cost,max-deployment-cost-per-hour:{amount:50,currency-code:USD}}" --target-vm-sizes "[Standard_NC48ads_A100_v4,Standard_ND96asr_v4]" --subscription-options "{include-quota:True,include-placement:True,include-pricing:True}"Identify--locationLocation.--modelModel identity, sizing inputs, and optional maximum model context length.--workloadInference request concurrency and token shape assumptions.--deploymentDeployment scale and Azure virtual machine procurement option.--optimizationRanking preference and optional deployment-wide hourly cost limit.--target-vm-sizesThe Azure virtual machine candidate set.--subscription-optionsThe subscription-scoped lookups to include in the recommendation.Options
--deploymentDeployment scale and Azure virtual machine procurement option.--modelModel identity, sizing inputs, and optional maximum model context length.--optimizationRanking preference and optional deployment-wide hourly cost limit.--subscription-optionsThe subscription-scoped lookups to include in the recommendation.--target-vm-sizesThe Azure virtual machine candidate set.--workloadInference request concurrency and token shape assumptions.