ARW
ARW-1 • Discovery Ready

Implement ARW in 2 Hours

This guide walks you through implementing ARW-1 (Discovery Ready)—the foundation for making your website agent-accessible. No complex setup, just three files.

Prerequisites

Node.js 18+ or Rust

For running validators and tools

Existing Website

Any static or dynamic site

30 min - 2 hours

Depending on site complexity

Implementation Steps

1

Create /llms.txt Manifest

This YAML file is the single source of truth for AI agents. It declares your content, actions, and policies.

File: /public/llms.txt (or /llms.txt at root)

# Agent-Ready Web Discovery Manifest
# https://yoursite.com/llms.txt

version: 1.0
profile: ARW-1

# Site Information
site:
  name: "Your Site Name"
  description: "Brief description of your site"
  homepage: "https://yoursite.com"
  contact: "ai@yoursite.com"

# Machine-Readable Content
# Note: Last-modified dates are in sitemap.xml per web standards
content:
  # Homepage
  - url: /
    machine_view: /index.llm.md
    purpose: homepage
    priority: high

  # About page
  - url: /about
    machine_view: /about.llm.md
    purpose: documentation
    priority: medium

  # Product or content pages
  - url: /products/example
    machine_view: /products/example.llm.md
    purpose: product_information
    priority: high
    chunks:
      - id: product-summary
        heading: "Product Overview"
      - id: product-specs
        heading: "Specifications"

# Usage Policies
policies:
  training:
    allowed: false
    note: "Content not licensed for model training"
  inference:
    allowed: true
    restrictions: ["attribution_required"]
  attribution:
    required: true
    format: "link"
    template: "Source: Your Site Name (https://yoursite.com)"
2

Create Machine Views (.llm.md)

Machine views are clean Markdown versions of your pages, optimized for AI agents. 85% smaller than HTML, no navigation or ads.

File: /public/index.llm.md

# Your Site Name

<!-- chunk: overview -->

## Overview

Clear, semantic content optimized for LLM parsing.
No navigation, ads, or clutter—just pure content.

We provide [product/service] for [target audience].

<!-- chunk: features -->

## Features

- **Feature 1**: Description of feature 1
- **Feature 2**: Description of feature 2
- **Feature 3**: Description of feature 3

<!-- chunk: contact -->

## Contact

- Email: contact@yoursite.com
- GitHub: https://github.com/yourusername
- Twitter: @yourhandle
💡 Pro Tip: Generate Automatically

Use the ARW CLI to generate machine views from your HTML:

npx arw@alpha generate ./pages --recursive
3

Add Discovery Headers & Link Tags

Add meta tags to your HTML `<head>` and HTTP headers for discovery and observability.

HTML Link Tags

<head>
  <!-- ARW Discovery -->
  <link rel="alternate" type="text/x-llm+markdown" href="/index.llm.md" />
  <link rel="alternate" type="application/yaml" href="/llms.txt" />
</head>

HTTP Headers (add to server config)

# For .llm.md files
Content-Type: text/x-llm+markdown; charset=utf-8
AI-Attribution: required
AI-Inference: allowed
AI-Training: not-allowed

# For /llms.txt
Content-Type: application/yaml; charset=utf-8
Example: Nginx Config
location ~ \.llm\.md$ {
    add_header Content-Type "text/x-llm+markdown; charset=utf-8";
    add_header AI-Attribution "required";
    add_header AI-Inference "allowed";
    add_header AI-Training "not-allowed";
}

location = /llms.txt {
    add_header Content-Type "application/yaml; charset=utf-8";
}
4

Validate Your Implementation

Use the ARW validator to check compliance and identify issues.

Option 1: Online Validator

npx arw-validator https://yoursite.com

Option 2: Local Validator

# Python validator
python tools/validators/validate-arw.py ./public/llms.txt

# Node.js validator
node tools/validators/validate-arw.mjs ./public/llms.txt schemas/arw_model.json

Option 3: ARW Inspector (Visual)

Open the ARW Inspector web tool and enter your URL:

Launch ARW Inspector

Expected Results

85%
Token Reduction

55KB HTML → 8KB Markdown

10x
Faster Discovery

2 seconds vs 5 minutes

$450
Monthly Savings

Typical medium site

🎉 Congratulations!

You've implemented ARW-1 (Discovery Ready). AI agents can now:

  • Discover your content via `/llms.txt` in seconds
  • Access efficient machine views (85% smaller)
  • Respect your usage policies
  • Provide attribution via headers

Next Steps

Progress to ARW-2 (Semantic Ready)

Add advanced features:

  • •Content chunking for precise citations
  • •Full AI-* header suite
  • •Rate limiting configuration

Test with ARW Inspector

Visualize your implementation:

  • View efficiency metrics
  • Check conformance level
  • Calculate ROI
Launch Inspector

Additional Resources

Full SpecificationCase StudiesCommunity Discussions