DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs

A 25-page preprint by Zeyu Cao, Xuan Guo, Cheng Zhang, Cheuk Hang Lau, Ilia Shumailov and Yiren Zhao of the University of Cambridge, the University of Oxford and Quettaflop AI, posted to arXiv on July 10, 2026 as 2608.14614 (cs.LG) with a PDF cover date of August 18, 2026. The paper reports the construction and one year of production operation of a 128-GPU cluster assembled entirely from second-hand components — the authors' "DumpsterCluster" — and asks under what conditions repurposing retired datacenter GPUs for LLM inference is economically viable and environmentally sustainable (Source: arxiv.org).

The paper's central finding is conditional in both directions. On capital cost the second-hand route is far cheaper: $22,000 for the 128-V100 pod as of March 2026 against $600,000 for an eight-GPU B200 system. On operating emissions it is far worse: under grid-average carbon intensity the authors measure approximately 4× higher total carbon per token for 8B models and over 40× for 70B models against current-generation hardware. The authors conclude that "GPU afterlife is not universally sustainable" and that hardware repurposing "must be strategically coupled with low carbon energy sources."

The hardware and its price

The authors focus on NVIDIA V100 16GB accelerators, two generations behind Hopper and three behind Blackwell, which entered secondary markets after successive waves of datacenter decommissioning. They note that NVIDIA shipped more than 3 million datacenter GPU chips in 2023 alone and that datacenter GPUs are typically phased out on roughly three-year cycles aligned with warranty periods, so millions of units enter the second-hand market with each cycle.

The paper fits second-hand price data from eBay listings to a stretched exponential decay model, P(t) = P₀·e^−(t/τ)^β. For the V100 16GB (May 2018 launch, $10,000 list) it obtains τ = 48.3 months and β = 2.43 (R² = 1.000); for the A100 40GB (May 2020 launch, $12,000 list) τ = 61.7 months and β = 4.66 (R² = 0.9997). Both fitted exponents exceed 1, which the authors read as accelerating depreciation. By March 2026 V100 prices had fallen to about $60 (99.4% depreciation) and A100 40GB prices to about $1,900 (84.2%). The authors caution that batch pricing is unobservable while devices are still deployed in hyperscaler datacenters, so their data points begin only three to four years after release.

The cluster itself comprises two pods of 128 V100s, each server node holding eight GPUs across two SXM baseboards with 300 GB/s bidirectional NVLink within sets of four, PCIe 3.0 switches, dual Intel Xeon 6138 sockets with 256 GB DRAM, and four 100G ConnectX-5 NICs on a 100 Gbps RDMA fabric plus a 25 Gbps ConnectX-4 for management traffic. GPUs were acquired at $215 in May 2024 and $170 in July 2024; the full 2024 build cost roughly $32,000 including servers and switches, which the authors observe is about the 2024 MSRP of a single H100. Motherboards, CPUs, memory and NICs were all bought second-hand.

Serving engine and throughput

The authors wrote a custom LLM serving engine in Rust implementing device-level pipeline parallelism, which they describe as a strategy not found in commodity serving systems. Pipeline-first parallelization distributes model segments across devices rather than sharding within layers, which they argue suits older hardware whose binding constraints are HBM capacity, HBM bandwidth, FLOPs and inter-GPU bandwidth (300 GB/s NVLink on V100 against 900 GB/s on H100).

Benchmarks use an input length of 100 tokens generating 1,024 output tokens, following the Azure LLM conversation dataset, across ShareGPT, prefill-heavy and decode-heavy workloads. B200 figures were profiled with vLLM v0.17; H100 and A100 with vLLM v0.6; V100 results come from the authors' own engine.

On the 8B prefill-heavy workload the 128-V100 pod reaches about 224,000 tokens per second, above the single B200's roughly 62,000, which the authors attribute to aggressive pipeline parallelization at scale rather than per-device capability. On ShareGPT the same cluster delivers 21,500 TPS against the B200's 44,000. On the 70B model the ordering reverses in absolute terms — 1,512 TPS against 37,045 for the B200 pod — but the authors emphasize that a 70B model does not fit on a single 16 GB V100 at all, so the comparison is between some throughput and none without device-level pipelining. Scaling from a 4-V100 pod to 128 GPUs yields roughly 7.6× throughput on the 8B model. The paper also records that the A100 80GB configuration, itself priced from a second-hand build, emerges as the most cost-effective option over an N-year horizon in the 70B decode-heavy comparison.

Cost of ownership by energy market

The paper models four energy supplies: the US mix (20% renewable), the China mix (30%), the Brazil mix (90%), and a Chinese wind farm as a 100%-renewable idealization that still carries about 10 g/kWh of residual lifecycle emissions. Electricity prices come from EIA (US), CEIC (China) and regional statistics (Brazil); carbon intensities and renewable shares come from 2023 grid emission factors via Ember and Our World in Data.

Total cost of ownership over three, five and eight years reverses by model size. For the 8B model the 128-V100 pod is the cheapest option across all four energy mixes, and under the China mix up to 6× more cost-effective than an H100 system. For the 70B model the advantage disappears: under the same China mix the pod's three-year cost per token is roughly 1.5× that of an H100 system, or 67% of the H100's cost efficiency. The authors attribute the reversal to older hardware's higher per-token operational cost at larger model scale. They note against these figures that Together AI listed 8B inference at $0.10 per million tokens and 70B at $0.88, so V100-based serving can remain profitable even at the higher operating cost.

Carbon accounting

Embodied carbon is tabulated per device from die area, process node and VRAM type: 9.78 kg CO₂e of logic-die carbon and 4.48 kg of HBM2 carbon for the V100 16GB, against 14.67 kg and 19.2 kg for the H100 80GB and an estimated 28.8 kg and 46.08 kg for the B200 (B200 figures are deduced from the closest available process node and marked as such in the paper). Annualized over a three-year cycle these give 4.73 kg/yr for the V100 against 11.21 kg/yr for the H100 and an estimated 24.96 kg/yr for the B200.

The paper reports V100 carbon footprints under two accounting conventions: "extended," which continues to amortize embodied carbon over the longer life, and "zero embodied," which treats a device that would otherwise have been landfilled as carrying no remaining embodied cost. The gap between them is small relative to the operational term. On the 8B prefill-heavy workload under the US mix, the 128-V100 pod emits 4.31 g CO₂ per million tokens (4.12 zero-embodied) against 1.15 for a B200 — about 3.7×. On the 70B workload the same comparison is 639.04 g (609.52 zero-embodied) against 15.30 — over 40×. Under the 100%-renewable supply the 8B figures converge to 0.31 g against 0.07, while the 70B gap remains wide at 46.45 g against 0.61.

The authors draw from this that hardware efficiency alone cannot deliver sustainability outcomes, that second-hand deployments must be co-located with clean generation to avoid worsening the footprint of AI services, and that second-hand clusters may be best suited to monetizing otherwise-curtailed renewable generation in regions with abundant low-carbon power.

Reliability in operation

Across one year of continuous operation on the 128-GPU pod the authors logged 58 service-impacting incidents. Eight (13.8%) were true hardware failures requiring replacement — two NIC, one GPU and five CPU — and 50 (86.2%) were recoverable in software, comprising 32 PCIe resets and 18 software-reasoned resets. The eight hardware failures across 128 devices correspond to roughly a 6.25% annual device failure rate, which the authors say is consistent with the 10% spare-capacity provisioning they adopted. They report binning and stress-testing used GPUs before deployment and using redundant capacity with load balancing. They caution that second-hand fleets carry no vendor support, requiring in-house diagnostic and repair capability or higher replacement rates.

Stated limitations

The paper names three. First, an adoption paradox in the market analysis: the economics rest on second-hand prices that reflect weak demand for older models, and broad uptake of the approach would raise those prices and erode the advantage documented. Second, the approach is specific to inference. Training requires high inter-GPU bandwidth for gradient synchronization, large batch sizes, precision guarantees and uninterrupted multi-week runs, all of which the authors argue the V100's 300 GB/s NVLink and higher failure rate would compromise; they note that inference nonetheless increasingly dominates deployment economics because a model is trained once and served many times. Third, all economic figures are pinned to March 2026 market prices and are sensitive to market conditions.

The authors also frame second-hand supply as a supply-chain argument independent of price: secondary markets are geographically distributed, less exposed to export controls on frontier technology, and available without the six-to-twelve-month procurement lead times seen during recent GPU shortages — relevant to organizations facing vendor prioritization of hyperscale customers or export restrictions (Export Controls (AI)).

Relationships

• supports: Inference Economics and Token Pricing — supplies measured cost-per-token and throughput figures for a hardware tier outside the frontier fleet
• supports: AI Environmental Impact — quantifies the embodied-versus-operational carbon trade-off in hardware reuse
• depends-on: AI Data Centers — hardware retirement cycles are the supply side of the secondary market
• related: Compute Governance, Export Controls (AI), Ilia Shumailov