WebSpark.HttpClientUtility v2.0.0 — nugetz
markhazleton/ WebSpark.HttpClientUtility v2.0.0 .NET 8.0 .NET 9.0 .NET 10.0
Drop-in HttpClient wrapper for .NET 8-10+ with Polly resilience, response
caching, and OpenTelemetry. One-line setup eliminates boilerplate for retries, circuit
breakers, correlation IDs, and structured logging. Perfect for microservices, APIs, and
web scrapers. Includes authentication providers (Bearer, Basic, API Key), concurrent
requests, fire-and-forget operations, and streaming support. MIT licensed, 252+ tests.
For web crawling features, see WebSpark.HttpClientUtility.Crawler package.
Get Started dotnet CLI PackageReference Package Manager
$ dotnet add package WebSpark.HttpClientUtilityA production-ready HttpClient wrapper for .NET 8+ that makes HTTP calls simple, resilient, and observable.
Stop writing boilerplate HTTP code. Get built-in resilience, caching, telemetry, and structured logging out of the box.
📦 v2.0 - Now in Two Focused Packages! Starting with v2.0, the library is split into two packages:
📚 Documentation The complete documentation site includes:
Getting started guide
Feature documentation
API reference
Code examples
Best practices
⚡ Quick Start
Core HTTP Features (Base Package) dotnet add package WebSpark.HttpClientUtility
// Program.cs - Register services (ONE LINE!)
builder.Services.AddHttpClientUtility();
// YourService.cs - Make requests
public class WeatherService
{
private readonly IHttpRequestResultService _httpService;
public WeatherService(IHttpRequestResultService httpService) => _httpService = httpService;
public async Task<WeatherData?> GetWeatherAsync(string city)
{
var request = new HttpRequestResult<WeatherData>
{
RequestPath = $"https://api.weather.com/forecast?city={city}",
RequestMethod = HttpMethod.Get
};
var result = await _httpService.HttpSendRequestResultAsync(request);
return result.IsSuccessStatusCode ? result.ResponseResults : null;
}
}
✅ Automatic correlation IDs for tracing
✅ Structured logging with request/response details
✅ Request timing telemetry
✅ Proper error handling and exception management
✅ Support for .NET 8 LTS and .NET 9
Web Crawling Features (Crawler Package) dotnet add package WebSpark.HttpClientUtility
dotnet add package WebSpark.HttpClientUtility.Crawler
// Program.cs
builder.Services.AddHttpClientUtility();
builder.Services.AddHttpClientCrawler(); // Adds crawler features
public class SiteAnalyzer
{
private readonly ISiteCrawler _crawler;
public SiteAnalyzer(ISiteCrawler crawler) => _crawler = crawler;
public async Task<CrawlResult> AnalyzeSiteAsync(string url)
{
var options = new CrawlerOptions
{
MaxDepth = 3,
MaxPages = 100,
RespectRobotsTxt = true
};
return await _crawler.CrawlAsync(url, options);
}
}
🎯 Why Choose This Library? Challenge Solution Boilerplate Code One-line service registration replaces 50+ lines of manual setup Transient Failures Built-in Polly integration for retries and circuit breakers Repeated API Calls Automatic response caching with customizable duration Observability Correlation IDs, structured logging, and OpenTelemetry support Testing All services are interface-based for easy mocking Package Size Modular design - install only what you need
🚀 Features
Base Package Features
Simple API - Intuitive request/response modelAuthentication - Bearer token, Basic auth, API key providersCorrelation IDs - Automatic tracking across distributed systemsStructured Logging - Rich context in all log messagesTelemetry - Request timing and performance metricsError Handling - Standardized exception processingType-Safe - Strongly-typed request and response modelsCaching - In-memory response caching (optional)Resilience - Polly retry and circuit breaker policies (optional)Concurrent Requests - Parallel request processingFire-and-Forget - Background request executionStreaming - Efficient handling of large responsesOpenTelemetry - Full observability integration (optional)CURL Export - Generate CURL commands for debugging
Crawler Package Features
Site Crawling - Full website crawling with depth controlRobots.txt - Automatic compliance with robots.txt rulesSitemap Generation - Create XML sitemaps from crawl resultsHTML Parsing - Extract links and metadata with HtmlAgilityPackSignalR Progress - Real-time crawl progress updatesCSV Export - Export crawl results to CSV filesPerformance Tracking - Monitor crawl speed and efficiency
📚 Common Scenarios
Enable Caching builder.Services.AddHttpClientUtility(options =>
{
options.EnableCaching = true;
});
// In your service
var request = new HttpRequestResult<Product>
{
RequestPath = "https://api.example.com/products/123",
RequestMethod = HttpMethod.Get,
CacheDurationMinutes = 10 // Cache for 10 minutes
};
Add Resilience (Retry + Circuit Breaker) builder.Services.AddHttpClientUtility(options =>
{
options.EnableResilience = true;
options.ResilienceOptions.MaxRetryAttempts = 3;
options.ResilienceOptions.RetryDelay = TimeSpan.FromSeconds(2);
});
All Features Enabled builder.Services.AddHttpClientUtilityWithAllFeatures();
🔄 Upgrading from v1.x
If You DON'T Use Web Crawling No code changes required! Simply upgrade:
dotnet add package WebSpark.HttpClientUtility --version 2.0.0
Your existing code continues to work exactly as before. All core HTTP features (authentication, caching, resilience, telemetry, etc.) are still in the base package with the same API.
If You DO Use Web Crawling Three simple steps to migrate:
Step 1 : Install the crawler package
dotnet add package WebSpark.HttpClientUtility.Crawler --version 2.0.0
Step 2 : Add using directive
using WebSpark.HttpClientUtility.Crawler;
Step 3 : Update service registration
// v1.x (old)
services.AddHttpClientUtility();
// v2.0 (new)
services.AddHttpClientUtility();
services.AddHttpClientCrawler(); // Add this line
That's it! Your crawler code (ISiteCrawler, SiteCrawler, SimpleSiteCrawler, etc.) works identically after these changes.
📖 Documentation
🎓 Sample Projects
BasicUsage - Simple GET/POST requestsWithCaching - Response caching implementationWithResilience - Retry and circuit breaker patternsConcurrentRequests - Parallel request processingWebCrawler - Site crawling example
🆚 Comparison to Alternatives Feature WebSpark.HttpClientUtility Raw HttpClient RestSharp Refit Setup Complexity ⭐ One line ⭐⭐⭐ Manual ⭐⭐ Low ⭐⭐ Low Built-in Caching ✅ Yes ❌ Manual ❌ Manual ⚠️ Plugin Built-in Resilience ✅ Yes ❌ Manual ❌ Manual ❌ Manual Telemetry ✅ Built-in ⚠️ Manual ⚠️ Manual ⚠️ Manual Type Safety ✅ Yes ⚠️ Partial ✅ Yes ✅ Yes Web Crawling ✅ Yes ❌ No ❌ No ❌ No .NET 8 LTS Support ✅ Yes ✅ Yes ✅ Yes ✅ Yes
🤝 Contributing
Fork the repository
Create a feature branch
Add tests for your changes
Ensure all tests pass
Submit a pull request
📊 Project Stats
252+ Unit Tests - 100% passingSupports .NET 8 LTS & .NET 9 MIT Licensed - Free for commercial useActive Maintenance - Regular updates
📦 Related Packages
📄 License This project is licensed under the MIT License - see the LICENSE file for details.
🔗 Links
Compatibility .NET 8.0
.NET 9.0
.NET 10.0
