Topic
#python
Loot, blog posts and adjacent themes connected to this topic. Follow the tag to keep it in your orbit.
Loot
More from this topic
A Python crawler for unstable or JavaScript-heavy sites, with selector healing, structured-data extraction, adaptive rate limiting, and an MCP server for agent-driven crawling. Use only for authorized scraping. Anansi is a Python web scraping toolkit designed for sites that change often or need browser rendering. It combines adaptive parsing, structured-data extraction, incremental crawling, proxy support, and an MCP server so an LLM or agent workflow can drive fetch, extract, crawl, pause, resume, export, and metrics actions. Why it is useful Self-healing selectors: stores selector confidence and attempts fallback strategies when a layout changes. Structured extraction first: pulls JSON-LD, Open Graph, and Microdata before relying on brittle CSS selectors. Browser upgrade path: can switch from HTTP fetching to Playwright rendering for JavaScript-heavy pages. Crawler durability: includes an async crawler, SQLite-backed queue, incremental recrawls, ETag/Last-Modified handling, and resumable jobs. Agent-ready interface: ships with an MCP server so compatible LLM tools can operate crawls through tool calls. Best fit Use Anansi when you need a resilient research or data-extraction crawler for websites you are allowed to access, especially where pages change structure or require JavaScript rendering. It is most relevant for developers building data pipelines, monitoring workflows, competitive research dashboards, or agentic browsing systems. Quick evaluation checklist Confirm the target website permits your intended crawling use case. Start with structured data extraction before custom selectors. Enable browser rendering only where HTTP fetching is insufficient. Keep adaptive rate limiting active and respect Retry-After responses. Use the MCP server when you want an agent to orchestrate crawl tasks instead of manually scripting every step. Source notes The GitHub repository describes Anansi as a self-healing web scraper with selector repair, browser rendering fallback, Chrome-like TLS fingerprinting, Pydantic validation, incremental crawling, and an MCP server. The project is written primarily in Python and is licensed under Apache-2.0.
Free
Review open
Text: AI-generated
AI-generated · Automatically published by LinkLoot. Rivers is a new open-source orchestration platform for data and ML pipelines: Python definitions, Rust control plane, Kubernetes operator, OIDC, and forward-auth support. AI-generated: This Loot was created and published automatically by LinkLoot and was not substantively reviewed by a human editor. Rivers is worth a bookmark if you build data or ML pipelines and want a Python authoring layer without putting the whole control plane on Python runtime behavior. The project defines assets and repositories in Python, then uses a Rust core for graph resolution, execution planning, scheduling, partition handling, and the web UI. What to try Define simple assets as Python functions and materialize them locally with pip install rivers. Use rivers dev MODULEPATH for a single-process local setup with embedded storage, scheduler, and UI. Evaluate it against Dagster-style asset orchestration needs where startup speed, typed execution planning, and Kubernetes deployment matter. Check the Kubernetes path if you need CodeLocation CRDs, one-pod-per-step execution, Helm charts, and a separate operator. Review the authentication guide before exposing the UI: Rivers documents none, native OIDC with PKCE, and forward-auth mode behind a trusted proxy. Good fit Rivers looks most useful for teams experimenting with data asset orchestration, ML pipeline scheduling, or internal pipeline platforms that want Python ergonomics plus a compiled Rust scheduler/control plane. It also has practical production hooks: SurrealDB-backed state, Server-Sent Events for UI updates, Kubernetes-native execution, backfills, sensors, schedules, concurrency limits, and pluggable IO. Caveats This is still a young project. The GitHub repository was created in May 2026 and the PyPI package is at 0.4.0, so treat it as an evaluation candidate rather than a drop-in replacement for mature orchestrators. The repository license is AGPL-3.0, which matters for internal platform and SaaS use. LinkLoot has not performed a hands-on review; verify installation, security posture, and operational behavior in a disposable environment before using it with production workloads.
Free
Review open
An open-source Python framework for generating, evaluating, and improving multi-agent workflows from goals and feedback. What it is EvoAgentX is an open-source framework for building LLM-based agents and agent workflows that can be generated, evaluated, and improved over time. Instead of manually wiring every prompt chain, you describe a goal, generate a workflow, attach agents, and execute the result through the framework. Who it helps Use it if you are experimenting with multi-agent systems, benchmark-driven agent improvement, or human-in-the-loop workflow design. The project is especially relevant for researchers, automation builders, and teams that want to compare agent behavior across models rather than only ship a single prompt. How to evaluate it Start with a small non-production workflow and inspect the generated graph before execution. Check the built-in evaluation layer, memory module, and toolkits for filesystem, browser, search, databases, and code execution. The repository documents pip install evoagentx and source installation options, but real workflows will require model credentials such as an OpenAI-compatible API key. Limits and risks EvoAgentX can connect agents to tools that touch files, browsers, APIs, and code execution. Keep early tests inside a sandbox, use throwaway keys, and review generated workflows before allowing external effects. Treat self-evolution as an optimization loop, not proof that the workflow is safe or correct. Sources Primary source: https://github.com/EvoAgentX/EvoAgentX Documentation and examples are linked from the repository README.
Free
Review open
Blog