A Model Context Protocol (MCP) server that provides a tool to scrape web content and convert it to markdown format.

• Web Scraping: Download HTML content from any URL
• CSS Selector Support: Extract specific content using CSS selectors
• Markdown Conversion: Convert HTML content to markdown format
• Multiple Transport Modes: Support for both stdio and HTTP interfaces
• Type Safety: Strongly typed tool handlers

go build .

The project is organized into reusable packages:

• mcp/ - MCP server implementation with scrape tool
• scrape/ - Web scraping functionality
• service/ - Service layer and value objects

Run the server in stdio mode for integration with MCP clients:

./contentserver-mcp
# or explicitly
./contentserver-mcp --stdio

Run the server as an HTTP service for remote access:

./contentserver-mcp --http :8080

This will start the MCP server on http://localhost:8080/mcp.

The server provides a scrape tool with the following parameters:

• url (required): The URL of the webpage to scrape
• selector (required): CSS selector to extract specific content

• ID selectors: #content, #main
• Class selectors: .article, .content
• Tag selectors: article, div, main

{
  "name": "scrape",
  "arguments": {
    "url": "https://example.com",
    "selector": "#content"
  }
}

curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
      "name": "scrape",
      "arguments": {
        "url": "https://example.com",
        "selector": "#content"
      }
    }
  }'

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "content": [
      {
        "type": "text",
        "text": "{\"markdown\":\"# Extracted Content\\n\\nThis is the markdown content...\"}"
      }
    ]
  }
}

The tool provides detailed error messages for common issues:

• Missing required parameters
• Invalid URLs
• Network errors
• HTML parsing errors
• Selector not found

go test ./...

go build -o contentserver-mcp .

The MCP package can be reused in other projects. See the examples/ directory for complete examples:

package main

import (
    "flag"
    "log"
    "github.com/foomo/contentserver-mcp/mcp"
    "github.com/mark3labs/mcp-go/server"
)

func main() {
    port := flag.String("port", "8080", "HTTP server port")
    flag.Parse()
    
    s := mcp.NewServer()
    httpServer := server.NewStreamableHTTPServer(s)
    
    log.Printf("Starting MCP HTTP server on port %s...", *port)
    if err := httpServer.Start(":" + *port); err != nil {
        log.Fatal(err)
    }
}

package main

import (
    "net/http"
    "github.com/foomo/contentserver-mcp/mcp"
    "github.com/mark3labs/mcp-go/server"
)

func main() {
    s := mcp.NewServer()
    httpServer := server.NewStreamableHTTPServer(s)
    
    mux := http.NewServeMux()
    mux.HandleFunc("/health", healthCheckHandler)
    mux.Handle("/mcp", httpServer)
    
    // Add custom middleware
    handler := loggingMiddleware(corsMiddleware(mux))
    
    http.ListenAndServe(":8080", handler)
}

See examples/ directory for complete working examples.

This project is part of the foomo content server ecosystem.