Officials from U.S. Special Operations Command are gearing up to assess and downselect industry hardware offerings that can support SOCOM’s growing use of AI and large language models.
The organization is looking to enter into procurement contracts or other agreements with vendors whose solutions are favorably evaluated by subject matter experts from SOCOM’s J24 Intelligence Data Science Team, according to a special notice about the effort.
The initiative comes as Defense Department components are keen on acquiring generative artificial intelligence tools — including large language models — to aid enterprise and warfighting tasks.
“The U.S. Government’s data science portfolio is rapidly expanding its reliance on large-scale AI workloads, especially LLMs and high-speed inference pipelines. To sustain this growth and to maintain a strategic edge, the program requires cutting-edge GPU acceleration, capable of delivering the throughput and memory bandwidth needed for state-of-the-art training, finetuning, and deployment. Advanced GPUs will provide a high-performance, energy-efficient, and future-ready foundation for advanced AI workloads, while ensuring low response times, reliability, and room for future growth,” officials wrote in the special notice about plans for hardware-enabled AI acceleration.
SOFWERX — an innovation hub located in Tampa, Florida, that connects SOCOM with innovators to help solve some of special operations forces’ most difficult challenges — is preparing to host a technology assessment event in collaboration with the J24 team.
The aim of the gathering is to “determine the best solution to upgrade a remote location with high-performance Graphics Processing Unit (GPU) servers to support large language model (LLM) workloads for up to 100+ concurrent users. The system will be fully turnkey, including all GPUs, memory, storage, networking, cooling, and power infrastructure, ready for immediate operation. This configuration ensures fast, reliable Artificial Intelligence (AI) performance today and scalability for future growth,” according to the notice.
Officials did not identify the remote location where the tech might be installed.
“Vendors must provide a turnkey solution that minimizes on-site assembly, configuration, and troubleshooting, ensuring the server is ready for immediate use with minimal IT intervention,” officials noted.
SOCOM wants to deploy advanced GPU hardware that offers a high bandwidth, energy-efficient capability to power LLM inference, finetuning, and retrieval-augmented generation workloads, which help optimize the output of large language models.
The command also seeks tech that can support GPU-to-GPU communication within the server and connectivity to the site’s broader network.
“The GPUs must be delivered as part of a complete, rack-mounted server solution suitable for immediate deployment in the data center at a remote site,” per the notice.
“The solution must support deployment to two networks, within air-gapped or otherwise strictly isolated environments. The server(s) and all GPUs shall be physically and logically isolated from each other, and any non-approved networks (no dual-homed network connections),” officials wrote.
Assessment criteria for vendor offerings include performance and scalability, infrastructure fit and reliability, cost and lifecycle considerations, and multi-network deployment and support, according to a SOFWERX list.
Performance metrics will include the ability to handle large-scale AI workloads, including LLM training, finetuning and high throughput inference; GPU-to-GPU communication bandwidth and latency within the server; overall compute throughput; and memory capacity and bandwidth to support “very large models.”
A Q&A session for potential offerors is slated for Dec. 3 via teleconference.
The deadline for solution submissions is Dec. 9, according to participation instructions posted by SOFWERX.
Those downselected will be invited to participate in an assessment event at SOFWERX in January.
link