FOR DEVELOPERS

Simple implementation, powerful results

Add llms.txt in 2 hours. Progressive enhancement means each step adds value without breaking existing functionality.

Why developers choose ARW

Progressive enhancement

Start with basic discovery and add features incrementally. Each step provides value without breaking existing functionality. No big-bang rewrite required.

Works with any stack

Next.js, WordPress, Rails, Django, static sites—ARW is just files and HTTP headers. No framework lock-in or special dependencies.

Built on standards

Uses HTTP, Markdown, YAML, OAuth 2.0, and Schema.org. No proprietary protocols or vendor-specific APIs to learn.

SEO unchanged

Machine views (.llm.md) are separate from HTML. Your SEO, social previews, and human experience remain untouched.

CLI tools

Rust-based CLI generates llms.txt from your sitemap, validates ARW files, and creates machine views. Automate the tedious parts.

Open source

MIT licensed specification and reference implementations. No fees, no lock-in, full control over your implementation.

Progressive implementation path

Time to first value: 2 hours. Full implementation: 2-3 months.

Basic Discovery

Week 1

Create llms.txt listing your top 20 pages with descriptions.

# /llms.txt
version: 0.1

site:
  name: "My Site"
  description: "Description"

content:
  - url: /docs/getting-started
    description: "Getting started guide"
    priority: high

Machine Views

Weeks 2-4

Create .llm.md files for top pages. Clean markdown without HTML overhead.

<!-- /docs/getting-started.llm.md -->

# Getting Started

<!-- chunk: overview -->
## Overview
Quick start guide for developers...

<!-- chunk: installation -->
## Installation
```bash
npm install my-package
```

Enhanced Discovery

Week 5

Add policies, metadata, and chunk details to llms.txt.

content:
  - url: /docs/getting-started
    machine_view: /docs/getting-started.llm.md
    priority: high
    chunks:
      - id: overview
        heading: "Overview"
      - id: installation
        heading: "Installation"

policies:
  training:
    allowed: true
  inference:
    allowed: true
  attribution:
    required: true

OAuth Actions

Weeks 6-10

Implement OAuth endpoints and declare actions in llms.txt.

actions:
  - id: create_ticket
    name: "Create Support Ticket"
    endpoint: /api/actions/create-ticket
    method: POST
    auth: oauth2
    scopes: ["support:write"]

oauth:
  authorization_url: /oauth/authorize
  token_url: /oauth/token

Quick start example

Create your first llms.txt in minutes.

/public/llms.txt

# Agent-Ready Web Discovery Manifest
version: 0.1

site:
  name: "Your Website"
  description: "Brief description of your site"
  contact: "ai@yoursite.com"
  homepage: "https://yoursite.com"

content:
  # Homepage
  - url: /
    machine_view: /.llm.md
    purpose: homepage
    priority: high

  # Documentation
  - url: /docs/getting-started
    machine_view: /docs/getting-started.llm.md
    purpose: technical_documentation
    priority: high
    chunks:
      - id: overview
        heading: "Overview"
      - id: installation
        heading: "Installation"

  # Product pages
  - url: /products/main-product
    machine_view: /products/main-product.llm.md
    purpose: product_information
    priority: high
    chunks:
      - id: product-summary
        heading: "Product Overview"
      - id: features
        heading: "Features"
      - id: pricing
        heading: "Pricing"

policies:
  training:
    allowed: false
    reasoning: "Content is proprietary"
  inference:
    allowed: true
    conditions: ["attribution_required"]
  attribution:
    required: true
    format: "YourSite <https://yoursite.com>"
  rateLimits:
    anonymous: "10/hour"
    authenticated: "100/hour"

Developer tools

ARW CLI

Rust-based command-line tool for generating, validating, and managing ARW files.

# Generate llms.txt from sitemap

$ arw generate --sitemap sitemap.xml

# Validate ARW files

$ arw validate

# Create machine view from HTML

$ arw convert page.html -o page.llm.md

Next.js Reference Implementation

Full working example with machine views, OAuth actions, and policy enforcement.

View on GitHub

Complete Specification

Detailed technical specification with CloudCart examples throughout.

Read the spec

Developer FAQs

Do I need to change my existing HTML?

No. Machine views (.llm.md) are parallel files. Your HTML, SEO, and user experience remain unchanged.

What about dynamic content?

Generate .llm.md files server-side just like HTML. Use the same data sources and keep them in sync with your existing rendering pipeline.

Can I use this with my CMS?

Yes. ARW works with any stack. For WordPress, Drupal, etc., add a plugin to generate machine views. For headless CMS, add .llm.md to your build process.

How do I handle authentication?

Use standard OAuth 2.0. If you already have OAuth (many SaaS apps do), just declare actions in llms.txt. If not, libraries exist for all major frameworks.

What's the maintenance burden?

Minimal. Once integrated into your build/render pipeline, machine views update automatically alongside HTML. llms.txt changes when you add/remove major sections.

Start building the agent-ready web

Progressive enhancement. Standards-based. Works with any tech stack. Get started in 2 hours.

Read the specification View examples