Topic

#ml-infrastructure

Loot, blog posts and adjacent themes connected to this topic. Follow the tag to keep it in your orbit.

#ml-infrastructure
1Shown loot
0Shown articles
7Linked neighbor tags
Topic paths

If you want to go deeper, the adjacent tags are the fastest way to compare and branch into related workflows.

#ai-generated#automated#data-sovereignty#hugging-face#public-interest#self-hosting#storage
Loot

More from this topic

Explore all loot
Self-host Hugging Face Xet storage without changing your tools

Self-host Hugging Face Xet storage without changing your tools

No votes yet
Text: AI-generated
#ai-generated#automated+6
AI-generated · Automatically published by LinkLoot. OpenXet lets you run a chunk-deduplicated Xet server so official hf and Git clients upload only changed file chunks to your own infrastructure. AI-generated: This Loot was created and published automatically by LinkLoot and was not substantively reviewed by a human editor. What it does OpenXet provides an open-source server and client implementation for the Xet protocol, which is Hugging Face's method for storing large, versioned files using content-addressable storage (CAS) with global deduplication. Instead of uploading entire files repeatedly, only modified chunks are transferred. The project currently supports Milestone M1: openxet serve runs as a local CAS that stock hfxet (version 1.7.0+) uploads to and downloads from directly. It includes verified uploads, scoped tokens, and signed download URLs. Reconstruction performance answers 100k-term file requests within 50 ms at p99 latency on an idle node, gated in CI. The system runs on local disk with a SQLite index. Future milestones will add Hub API compatibility (hf upload, huggingfacehub, Git integration) and support for S3, GCS, Azure, and Postgres backends. Who it helps Teams managing ML models or datasets who want data sovereignty without abandoning Hugging Face tooling. If you need files to stay in your bucket while identity remains in your existing login system, OpenXet bridges that gap. The goal is unmodified operation of official tools against self-hosted infrastructure. Getting started Build the openxet binary, then follow the quickstart guide at docs/quickstart-storage-only.md. Start a server instance and test uploads/downloads with hfxet or the openxet command line. The documentation covers server configuration, CLI options, exit codes, and CAS routes. Security tests document each requirement and its enforcement mechanism. Benchmarks detail the latency gate and reconstruction metrics. Note: There is no release yet. This is development-stage software. Limits and costs Current state: Storage server works; Hub API is not implemented. Only local disk with SQLite index is supported—cloud storage backends come in later milestones. No hosted service exists yet despite the openxet.com homepage reference. Apache-2.0 licensing permits commercial use but does not imply free hosting. Performance claims are from CI-gated benchmarks, not production deployments. Source links Official repository: https://github.com/stubbi/openxet Project documentation: https://github.com/stubbi/openxet/blob/main/README.md Project license: https://github.com/stubbi/openxet/blob/main/LICENSE
LinkLoot access
Free
Provider costs
Unknown
Review open
0
Open loot
Blog

Related reads

Browse blog
No blog posts for #ml-infrastructure yet

There is no published article with this tag right now. Browse the blog for adjacent themes or follow the tag for future updates.