Intel's D270 Has 160GB on Intel's Card, Up to 480GB From Partners

Developers planning around Intel's new 480GB D270 headline need to check the board vendor. Intel's own PCIe card has 160GB of memory. The design lets manufacturing partners build cards with up to 480GB, according to a footnote in Intel's launch announcement. That 320GB difference changes how much room remains for a model's working memory before the first user sends a request.

The D270 is an inference accelerator for running deployed AI models. Its product page lists a 350W air-cooled design and a standard PCIe 5.0 interface. Intel is targeting long prompts and concurrent AI sessions in existing data centers. For a developer choosing a server, the useful question is how many requests the installed card can handle at an acceptable response time. Maximum memory capacity answers only part of that.

Read the memory configuration before the model claim

Intel presented the D-series, previously called Crescent Island, at the Open Compute Project Global Summit on October 12. Its announcement supplies these figures. NVIDIA's RTX PRO 6000 Blackwell Server Edition specifications provide a comparison:

The maximum partner configuration has five times NVIDIA's listed memory capacity. Intel's own configuration has about 1.67 times as much. Yet the advertised bandwidth is slightly lower than NVIDIA's 1.597TB/s. Those ratios are simple calculations from the vendors' specifications, without an inference benchmark behind them. NVIDIA also lists a configurable maximum power consumption of 600W and an air-cooled dual-slot option on its server card page. Air cooling alone therefore doesn't distinguish Intel's approach. The larger memory pool within its stated power envelope is the more useful proposition to test.

Loading the weights leaves another memory bill

Hugging Face's model memory estimator makes a distinction that applies to any accelerator: its estimate covers loading a model, while inference needs additional memory. It can calculate sizes for different data types without downloading the full weights. That makes it useful for checking an initial capacity assumption before arranging access to a new board.

Consider a hypothetical model with exactly 70 billion parameters, all stored at 16 bits each. The weights alone occupy 140 billion bytes, or 140GB in decimal units. On a nominal 160GB card, that leaves about 20GB before other allocations. This is our arithmetic illustration, not a measured D270 configuration or a claim that a particular 70B model will run. The estimator's loading-versus-inference caveat explains why the distinction matters. A model that fits can still leave too little space for the context lengths and request concurrency an application needs.

The extra allocation includes the key-value, or KV, cache. It stores attention data from earlier tokens so the model can reuse it while generating a response. Transformers' cache documentation describes a dynamic cache that grows as generation progresses. Holding more sessions in memory can therefore consume capacity that a weights-only calculation leaves apparently free. Some model architectures limit cache growth with sliding or chunked attention, so context length doesn't translate into the same memory requirement for every model.

The same documentation explains the choices when memory runs short. Offloading moves most cache storage to the CPU, with transfers during computation. Quantizing the cache reduces its precision and memory footprint, but can hurt latency for short contexts when sufficient GPU memory is already available. A bigger board may let a workload avoid those compromises. Whether it does depends on the cache implementation and traffic, not simply the parameter count printed beside a model's name.

Quantization changes the comparison

Weight precision can alter the purchase calculation before cache storage enters it. Hugging Face's bitsandbytes guide describes 8-bit loading as halving memory usage and also documents 4-bit loading. These formats reduce the space occupied by weights. They don't turn the advertised memory capacity of a card into a fixed upper limit on model parameters that applies to every deployment.

The guide also separates the stored weight format from the data type used for computation. Other modules can retain a different precision, and its examples let developers set the compute type separately. For a D270 evaluation, that means recording the actual model format and execution settings alongside throughput. bitsandbytes lists Intel XPU support, but that general compatibility statement isn't a D270 performance result. A comparison using different quantization settings could attribute a software choice to the new hardware.

More storage doesn't settle response speed

An AI request has different phases. Prefill processes the incoming prompt, while decode generates the response token by token. NVIDIA's technical analysis of long-context attention describes dense-attention prefill as compute-bound and ordinary decode as constrained by memory reads. It also explains exceptions: speculative decoding changes the amount of parallel work, and a short new prompt attending to a large cached prefix can behave more like decode.

Those distinctions limit what the D-series specifications can tell you. Extra capacity can accommodate more stored state, but the rate of reading it remains a separate constraint. NVIDIA's analysis also ties attention performance to model architecture and sequence length. The engineering inference is that Intel's larger memory pool needs workload-specific tests. A long document submitted once and an agent repeatedly extending a cached conversation needn't stress the card in the same way, even if both fit comfortably in memory.

Check the serving stack as carefully as the board

Intel's D270 page names vLLM and PyTorch as part of its software pitch. It also says the design reallocates die area from graphics and ray-tracing pipelines to compute. That focuses the hardware story on inference. The remaining question for application teams is whether their chosen model and serving features work on the supplied software stack.

There is an existing Intel path in vLLM's GPU installation documentation. The XPU backend supports Intel data-center and Arc GPUs and depends on a separate package, vllm-xpu-kernels. Its installation instructions specify Python 3.12 for the provided kernel wheel. That is concrete upstream support, with dependencies a deployment team has to account for.

The vLLM instructions also document XPU tensor-parallel serving and describe pipeline parallelism as a beta feature with a single-node multiprocessing backend. These details matter when moving beyond one card. They establish a route for serving on Intel GPUs, while leaving D270-specific validation to the actual deployment. A useful trial should preserve the application's model revision and serving settings so failures or gains can be traced. Changing the board, model format and runtime together would make the result harder to explain.

Measure the wait at the intended load

NVIDIA's inference benchmarking documentation distinguishes time to first token from the interval between subsequent tokens. The first includes queuing, prompt processing and network latency. The second describes generation after output begins. Both affect whether a coding assistant or document service feels responsive, and neither can be read directly from gigabytes of memory.

It also distinguishes total system throughput from throughput per user. As concurrency rises, aggregate output can increase while each user's response slows. For a D270 trial, we'd report the installed memory configuration alongside those latency measures, using the same prompt and output lengths across comparisons. The benchmarking guide warns that tools define some metrics differently. Matching definitions is necessary before treating a larger tokens-per-second number as a better service. Testing at the application's intended load would reveal whether extra resident sessions deliver useful work or merely lengthen the queue.

Intel says customer sampling is underway. The next evidence to watch is a named partner board with its exact memory configuration and reproducible serving results. Tests that report latency at increasing concurrency would show whether the additional capacity pays off. Until those results arrive, server sizing should start with the memory on the specified board, rather than assume every D270 offers the family's 480GB ceiling.