WebSpark.HttpClientUtility v2.1.2 — nugetz
markhazleton/ WebSpark.HttpClientUtility v2.1.2 .NET 8.0 .NET 9.0 .NET 10.0
Drop-in HttpClient wrapper for .NET 8-10+ with Polly resilience, response
caching, and OpenTelemetry. One-line setup eliminates boilerplate for retries, circuit
breakers, correlation IDs, and structured logging. Perfect for microservices, APIs, and
web scrapers. Includes authentication providers (Bearer, Basic, API Key), concurrent
requests, fire-and-forget operations, and streaming support. MIT licensed, 252+ tests.
For web crawling features, see WebSpark.HttpClientUtility.Crawler package.
Get Started dotnet CLI PackageReference Package Manager
$ dotnet add package WebSpark.HttpClientUtility
Stop writing 50+ lines of HttpClient setup. Get enterprise-grade resilience (retries, circuit breakers), intelligent caching, structured logging with correlation IDs, and OpenTelemetry tracing in a single AddHttpClientUtility() call. Perfect for microservices, background workers, and web scrapers.
🚀 Why Choose WebSpark.HttpClientUtility? Your HTTP setup in 1 line vs. 50+
Feature WebSpark.HttpClientUtility Raw HttpClient RestSharp Refit Setup Complexity ⭐ One line ⭐⭐⭐ 50+ lines manual ⭐⭐ Low ⭐⭐ Low Built-in Retry/Circuit Breaker ✅ Polly integrated ❌ Manual Polly setup ❌ Manual ❌ Manual Response Caching ✅ Configurable, in-memory ❌ Manual ❌ Manual ❌ Manual Correlation IDs ✅ Automatic ❌ Manual middleware ❌ Manual ❌ Manual OpenTelemetry ✅ Built-in ❌ Manual ActivitySource ❌ Manual ❌ Manual Structured Logging ✅ Rich context ❌ Manual ILogger ⭐⭐ Basic ⭐⭐ Basic Web Crawling ✅ Separate package ❌ No ❌ No ❌ No Production Trust ✅ 252+ tests, LTS support ✅ Microsoft-backed ✅ Popular (7M+ downloads) ✅ Popular (10M+ downloads)
✅ Building microservices with distributed tracing requirements
✅ Need resilience patterns without writing Polly boilerplate
✅ Want intelligent caching for API rate-limit compliance
✅ Building web scrapers or crawlers (with Crawler package)
When NOT to use WebSpark:
❌ You need declarative, type-safe API clients (use Refit)
❌ You want maximum control and minimal magic (use raw HttpClient)
❌ Legacy .NET Framework 4.x projects (WebSpark requires .NET 8+)
🛡️ Production Trust Battle-Tested & Production-Ready
✅ 252+ unit tests with 100% passing - tested on .NET 8, 9, and 10
✅ Continuous Integration via GitHub Actions - every commit tested
✅ Semantic Versioning - predictable, safe upgrades
✅ Zero breaking changes within major versions - backward compatibility guaranteed
✅ Framework Support: .NET 8 LTS (supported until Nov 2026), .NET 9, .NET 10 (Preview)
✅ MIT Licensed - free for commercial use
🔄 Active development - regular updates and improvements
📅 Long-term support - each major version supported for 18+ months
💬 Community support - GitHub Discussions for questions and best practices
📖 Comprehensive documentation - Full docs site
Breaking Change Commitment
We follow semantic versioning strictly:
Patch versions (2.0.x): Bug fixes only, zero breaking changesMinor versions (2.x.0): New features, backward compatibleMajor versions (x.0.0): Breaking changes with detailed migration guides
📦 v2.0 - Now in Two Focused Packages! Starting with v2.0, the library is split into two packages:
📚 Documentation The complete documentation site includes:
Getting started guide
Feature documentation
API reference
Code examples
Best practices
⚡ 30-Second Quick Start dotnet add package WebSpark.HttpClientUtility
Minimal Example (Absolute Minimum)
// Program.cs
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddHttpClientUtility();
var app = builder.Build();
app.MapGet("/weather", async (IHttpRequestResultService http) =>
{
var request = new HttpRequestResult<WeatherData>
{
RequestPath = "https://api.weather.com/forecast?city=Seattle",
RequestMethod = HttpMethod.Get
};
var result = await http.HttpSendRequestResultAsync(request);
return result.IsSuccessStatusCode ? Results.Ok(result.ResponseResults) : Results.Problem();
});
app.Run();
record WeatherData(string City, int Temp);
✅ Automatic correlation IDs for tracing
✅ Structured logging with request/response details
✅ Request timing telemetry
✅ Proper error handling and exception management
✅ Support for .NET 8 LTS, .NET 9, and .NET 10 (Preview)
📖 Show more: Service-based pattern with error handling
// Program.cs
builder.Services.AddHttpClientUtility(options =>
{
options.EnableCaching = true; // Cache responses
options.EnableResilience = true; // Retry on failure
});
// WeatherService.cs
public class WeatherService
{
private readonly IHttpRequestResultService _http;
private readonly ILogger<WeatherService> _logger;
public WeatherService(
IHttpRequestResultService http,
ILogger<WeatherService> logger)
{
_http = http;
_logger = logger;
}
public async Task<WeatherData?> GetWeatherAsync(string city)
{
var request = new HttpRequestResult<WeatherData>
{
RequestPath = $"https://api.weather.com/forecast?city={city}",
RequestMethod = HttpMethod.Get,
CacheDurationMinutes = 10 // Cache for 10 minutes
};
var result = await _http.HttpSendRequestResultAsync(request);
if (!result.IsSuccessStatusCode)
{
_logger.LogError(
"Weather API failed: {StatusCode} - {Error}",
result.StatusCode,
result.ErrorDetails
);
return null;
}
return result.ResponseResults;
}
}
📖 Show more: Full-featured with auth and observability
// Program.cs - Advanced configuration
builder.Services.AddHttpClientUtility(options =>
{
options.EnableCaching = true;
options.EnableResilience = true;
options.ResilienceOptions.MaxRetryAttempts = 3;
options.ResilienceOptions.RetryDelay = TimeSpan.FromSeconds(2);
options.DefaultTimeout = TimeSpan.FromSeconds(30);
});
// WeatherService.cs - Advanced usage
public async Task<WeatherData?> GetWeatherWithAuthAsync(string city, string apiKey)
{
var request = new HttpRequestResult<WeatherData>
{
RequestPath = $"https://api.weather.com/forecast?city={city}",
RequestMethod = HttpMethod.Get,
CacheDurationMinutes = 10,
Headers = new Dictionary<string, string>
{
["X-API-Key"] = apiKey,
["Accept"] = "application/json"
}
};
var result = await _http.HttpSendRequestResultAsync(request);
// Correlation ID is automatically logged and propagated
_logger.LogInformation(
"Weather request completed in {Duration}ms with correlation {CorrelationId}",
result.RequestDuration,
result.CorrelationId
);
return result.IsSuccessStatusCode ? result.ResponseResults : null;
}
Web Crawling Features (Crawler Package) dotnet add package WebSpark.HttpClientUtility
dotnet add package WebSpark.HttpClientUtility.Crawler
// Program.cs
builder.Services.AddHttpClientUtility();
builder.Services.AddHttpClientCrawler(); // Adds crawler features
public class SiteAnalyzer
{
private readonly ISiteCrawler _crawler;
public SiteAnalyzer(ISiteCrawler crawler) => _crawler = crawler;
public async Task<CrawlResult> AnalyzeSiteAsync(string url)
{
var options = new CrawlerOptions
{
MaxDepth = 3,
MaxPages = 100,
RespectRobotsTxt = true
};
return await _crawler.CrawlAsync(url, options);
}
}
🚀 Features
Base Package Features
Simple API - Intuitive request/response modelAuthentication - Bearer token, Basic auth, API key providersCorrelation IDs - Automatic tracking across distributed systemsStructured Logging - Rich context in all log messagesTelemetry - Request timing and performance metricsError Handling - Standardized exception processingType-Safe - Strongly-typed request and response modelsCaching - In-memory response caching (optional)Resilience - Polly retry and circuit breaker policies (optional)Concurrent Requests - Parallel request processingFire-and-Forget - Background request executionStreaming - Efficient handling of large responsesOpenTelemetry - Full observability integration (optional)CURL Export - Generate CURL commands for debugging
Crawler Package Features
Site Crawling - Full website crawling with depth controlRobots.txt - Automatic compliance with robots.txt rulesSitemap Generation - Create XML sitemaps from crawl resultsHTML Parsing - Extract links and metadata with HtmlAgilityPackSignalR Progress - Real-time crawl progress updatesCSV Export - Export crawl results to CSV filesPerformance Tracking - Monitor crawl speed and efficiency
📚 Common Scenarios
Enable Caching builder.Services.AddHttpClientUtility(options =>
{
options.EnableCaching = true;
});
// In your service
var request = new HttpRequestResult<Product>
{
RequestPath = "https://api.example.com/products/123",
RequestMethod = HttpMethod.Get,
CacheDurationMinutes = 10 // Cache for 10 minutes
};
Add Resilience (Retry + Circuit Breaker) builder.Services.AddHttpClientUtility(options =>
{
options.EnableResilience = true;
options.ResilienceOptions.MaxRetryAttempts = 3;
options.ResilienceOptions.RetryDelay = TimeSpan.FromSeconds(2);
});
All Features Enabled builder.Services.AddHttpClientUtilityWithAllFeatures();
🔄 Upgrading from v1.x
If You DON'T Use Web Crawling No code changes required! Simply upgrade:
dotnet add package WebSpark.HttpClientUtility --version 2.0.0
Your existing code continues to work exactly as before. All core HTTP features (authentication, caching, resilience, telemetry, etc.) are still in the base package with the same API.
If You DO Use Web Crawling Three simple steps to migrate:
Step 1 : Install the crawler package
dotnet add package WebSpark.HttpClientUtility.Crawler --version 2.0.0
Step 2 : Add using directive
using WebSpark.HttpClientUtility.Crawler;
Step 3 : Update service registration
// v1.x (old)
services.AddHttpClientUtility();
// v2.0 (new)
services.AddHttpClientUtility();
services.AddHttpClientCrawler(); // Add this line
That's it! Your crawler code (ISiteCrawler, SiteCrawler, SimpleSiteCrawler, etc.) works identically after these changes.
📖 Documentation
🎓 Sample Projects
BasicUsage - Simple GET/POST requestsWithCaching - Response caching implementationWithResilience - Retry and circuit breaker patternsConcurrentRequests - Parallel request processingWebCrawler - Site crawling example
🤝 Contributing
Fork the repository
Create a feature branch
Add tests for your changes
Ensure all tests pass
Submit a pull request
📊 Project Stats
252+ Unit Tests - 100% passingSupports .NET 8 LTS, .NET 9, & .NET 10 (Preview) MIT Licensed - Free for commercial useActive Maintenance - Regular updates
📦 Related Packages Testing Package Features:
FakeHttpResponseHandler - Mock HTTP responses without network callsFluent API - Easy test setup with ForRequest().RespondWith()Sequential Responses - Test retry behavior with multiple responsesRequest Verification - Assert requests were made correctlyLatency Simulation - Test timeout scenarios dotnet add package WebSpark.HttpClientUtility.Testing
📄 License This project is licensed under the MIT License - see the LICENSE file for details.
🔗 Links
Compatibility .NET 8.0
.NET 9.0
.NET 10.0
