Magic endpoint is a dynamic Hyperlambda endpoint evaluator, allowing you to create HTTP REST API endpoints dynamically, that will execute a Hyperlambda file when evaluated, where the URL is a reference to the physical path on disc to your Hyperlambda file. To use package go to https://polterguy.github.io
magic.endpoint is a dynamic endpoint URL controller, allowing you to declare endpoints that are dynamically
resolved using your IHttpExecutorAsync service implementation. The default implementation of this interface,
is the class called HttpExecutorAsync, and the rest of this file will be focused on documenting this
implementation, since it is the default service implementation for magic.endpoint - Although, technically, you
could exchange this with your own implementation if you wish, completely changing the behaviour of the library
if you wish to for instance resolve endpoints to Python, Ruby, or any other dynamic programming language
implementation, and you have some means to execute such code from within a .Net 6 environment.
The resolver will be invoked for all relative URLs starting with "magic/", for the following verbs.
GET
POST
PUT
DELETE
PATCH
The default service implementation will resolve everything after the "magic/" parts in the
given URL, to a Hyperlambda file assumed to be found relatively beneath your "/files/" folder -
Although, exactly where you physically put your files on disc, can be configured
through your "appsettings.json" file. The HTTP verb is assumed to be the last parts of your
filename, before its extension, implying an HTTP request such as the following.
GET magic/modules/foo/bar
Will resolve to the following physical file on disc.
files/modules/foo/bar.get.hl
Only the "magic" part of your URL is rewritten before the verb is appended to the URL, and
finally the extension ".hl" appended. Then the file is loaded and parsed as Hyperlambda, and whatever
arguments you pass in, either as query parameters or as your JSON payload is appended into your
resulting lambda node's [.arguments] node as arguments to your Hyperlambda file invocation.
The resolver will never return files directly, but is only able to execute Hyperlambda files,
so by default there is no way to get static files, unless you create a Hyperlambda endpoint that returns
a static file somehow.
The default resolver will only allow the client to resolve files inside your "/files/modules/"
folder and "/files/system/" folder. This allows you to safely keep files that parts of your system
relies upon inside your dynamic "/files/" folder, without accidentally creating endpoints clients can
resolve, resulting in breaches in your security. Only characters a-z, 0-9 and '-', '_' and '/' are legal
characters for the resolvers, and only lowercase characters to avoid file system incompatibilities between
Linux and Windows. There is one exception to this rule though, which is that the resolver will resolve
files and folder starting out with a period (.) character, since this is necessary to allow for having
"hidden files" being resolved as endpoints - Which is a requirement to make things such as
Apple's ".well-known" endpoints being resolved.
Below is probably the simplest HTTP endpoint you could create. Save the following Hyperlambda in a
file at the path of /modules/tutorials/foo.get.hl using for instance your Magic
"Hyper IDE" menu item.
return
result:Hello from Magic Backend
Then invoke the endpoint using the GET verb with the following URL.
http://localhost:5000/magic/modules/tutorials/foo
Arguments
The default IHttpExecutorAsync implementation can explicitly declare what arguments the file can
legally accept, and if an argument is given during invocation that the file doesn't allow for, an
exception will be thrown and the file will never be executed. This allows you to declare what
arguments your Hyperlambda file can accept, and avoid having anything but arguments explicitly
declared in your Hyperlambda file from being sent into your endpoint during invocation of your
HTTP endpoint. An example Hyperlambda file taking two arguments can be found below.
If you save this file on disc as /files/modules/tutorials/foo2.get.hl, you can invoke it as follows
using the HTTP GET verb - Assuming your backend is running on localhost at port 5000.
JSON payloads and form URL encoded payloads are automatically converted to lambda/nodes -
And query parameters are treated indiscriminately the same way as JSON payloads -
Except of course, query parameters cannot pass in complex graph objects, but only
simply key/value arguments. Only POST, PUT and PATCH endpoints can handle payloads. If you
supply a payload to a GET or DELETE endpoint, an exception will be thrown, and an error
returned to the caller.
To allow for any arguments to your files, simply ommit the [.arguments] node
in your Hyperlambda althogether, or supply an [.arguments] node and set its value to *.
Alternatively, you can also partially ignore arguments sanity checking of individual nodes,
by setting their values to *, such as the following illustrates.
.arguments
arg1:string
arg2:date
arg3:*
In the above arguments declaration, [arg1] and [arg2] will be sanity checked, and input converted
to string or date (DateTime) - But the [arg3] parts will be completely ignored, allowing the caller
to invoke it with anything as arg3 during invocation - Including complete graph JSON objects, assuming
the above declaration is for a PUT, POST or PATCH Hyperlambda file. The '*' value for an argument also turn
off all conversion, implying everything will be given to your lambda object with the JSON type the argument
was passed in as.
All arguments declared are considered optional, and the file will still resolve if the argument is not given,
except of course the argument won't exist in the [.arguments] node. However, no argument not found
in your [.arguments] declaration can be provided during invocations, assuming you choose to declare
an [.arguments] collection in your Hyperlambda endpoint file, and you don't set its value to *.
To declare what type your arguments can be, set the value of the argument declaration node to
the Hyperlambda type value inside of your arguments declaration, such as illustrated above.
Arguments will be converted if possible, to the type declaration in your argument's declaration.
If no conversion is possible, an exception will be thrown.
Although the sanity check will check graph objects, passed in as JSON payloads, it has its restrictions,
such as not being able to sanity check complex objects passed in as arrays, etc. If you need stronger
sanity checking of your arguments, you will have to manually check your more complex graph objects
yourself in your own Hyperlambda files.
Also realise that if the value originates from a payload, as in from a PUT, PATCH or POST JSON object
for instance, these types of objects might contain null values. If they do, no conversion will be attempted,
and internally within your endpoint's Hyperlambda code, you might therefor expect to see for instance
long values being in fact null, even though technically these are not nullable types in .Net.
Accepted Content-Type values
The POST, PUT and PATCH endpoints can intelligently handle any of the following Content-Types.
application/json
application/x-json
application/www-form-urlencoded
application/x-www-form-urlencoded
multipart/form-data
JSON types of payloads are fairly well described above, and URL encoded form payloads are handled
the exact same way, except of course the [.arguments] node is built from URL encoded values instead
of JSON - However, internally this is transparent for you, and JSON, query parameters, URL encoded
forms, and "multipart/form-data" can be interchanged 100% transparently from your code's perspective -
Except "multipart/form-data" might have [file] arguments wrapping streams that you need to
handle separately as such. File attachments will be passed into your endpoint as follows.
All other types of payloads will be passed in as the raw stream, not trying to read from it in any
ways, allowing you to intercept reading with things such as authentication, authorisation, logic of
where to persist content, etc. To understand how you can handle these streams, check out
the "magic.lambda.io" project's documentation, and specifically the [io.stream.xxx] slots.
Extending the Content-Type request and response resolver
The Content-Type resolver/parser is extendible, allowing you to change its behaviour by providing
your own callback that will be invoked for some specific Content-Type value provided. This is useful
if you want to be able to for instance handle "text/xml" or "text/csv" types of request/response
objects, and intelligently and automatically create an argument collection from it. Below is example
code illustrating how to create your own HTTP request resolver for the MIME type of "application/x-foo".
EndpointController.RegisterContentType("application/x-foo", async (signaler, request) =>
{
var args = new Node();
/* ... Create some sort of collection of arguments and put into args node here ... */
return args;
});
Notice - The argument sanity checking will still be invoked with a custom handler, implying
your Content-Type handler and the [.arguments] declaration in your Hyperlambda file still
needs to agree upon the arguments, and if a non-valid argument is specified to a Hyperlambda file,
an exception will be thrown. Also notice that registering a custom Content-Type is not thread
safe, and should be done as you start your application, and not during its life time.
You can also provide your own HTTP response resolver that will be invoked given some specified
Content-Type from your Hyperlambda file. This is done in a similar manner using something resembling
the following.
EndpointController.RegisterContentType("application/x-foo", (response) =>
{
/* ... Return some sort of IActionResult here ... */
return new ObjectResult(response.Content) { StatusCode = response.Result };
});
The above method should also exclusively be used during startup, and not later,
since it is not thread safe. The above method assumes you register your Content-Type handlers
as your application is starting.
Meta information
Due to the semantic structure of Hyperlambda, retrieving meta information from your HTTP endpoints
using this module is very easy. The project has one slot called [endpoints.list] that returns
meta information about all your endpoints. This slot again can be invoked using the following URL.
http://localhost:5000/magic/system/endpoints/list
This endpoint/slot will semantically traverse your endpoints, recursively loading up all Hyperlambda
files from disc that are resolved from a valid URL, and return meta information about the file/endpoint
back to the caller. This allows the system to easily figure out things such as the following about
your endpoints.
What is the endpoint's HTTP VERB
What is the endpoint's URL
What arguments can the endpoint handle
Has the file been given a friendly description, through a [.description] node
Etc ...
This slot/endpoint is what allows you to see meta information about all your HTTP REST endpoints
in the "Endpoints" menu item in the Magic dashboard for instance. The return value from this
slot/endpoint again, is what's used as some sort of frontend is being generated using the Magic
dashboard.
Extending the meta data retrieval process
You can extend the meta data retrieval process by
invoking ListEndpoints.AddMetaDataResolver, and pass in your own function. This class can be
found in the magic.endpoint.services.slots namespace.
The AddMetaDataResolver method takes one function object, which will be invoked for every file
the meta generator is trying to create meta data for, with the complete lambda, verb and args
of your endpoint. This allows you to semantically traverse the lambda/args nodes, and append
any amount of (additional) meta information you wish - Allowing you to extend the generating
of meta data, if you have some sort of general custom Hyperlambda module, creating custom
HTTP endpoints of some sort.
This function will be invoked for every single Hyperlambda file in your system,
every time meta data is retrieved, so you might want to ensure it executes in a fairly short
amount of time, not clogging the server or HTTP endpoint meta generating process in any ways.
Slots related to plugin
In addition to the meta retrieval endpoint described above, the module contains the following
slots.
[server.ip] - Returns the IP address of the server itself
[response.status.set] - Sets the status code (e.g. 404) on the response object
[request.cookies.list] - Lists all HTTP request cookies
[request.cookies.get] - Returns the value of a cookie sent by the request
[response.cookies.set] - Creates a cookie that will be returned to the client over the response
[request.headers.list] - Lists all HTTP request headers sent by the request
[request.headers.get] - Returns a single HTTP header associated with the request
[request.ip] - Returns the IP address of the HTTP request
[request.host] - Returns the host name associated with the request
[request.scheme] - Returns the scheme associated with the request
[response.headers.set] - Adds an HTTP header to the response object
Misc
Unless you explicitly change the Content-Type of your response object, by using
the [response.headers.set] slot, a Content-Type of application/json will be assumed,
and this header will be added to the resulting HTTP response object. If you wish to override
this behavious and return plain text for instance, you could create an endpoint containing
the following.
response.headers.set
Content-Type:text/plain
return:Hello from Magic Backend
If you intend to return anything but JSON, you must set the Content-Type header, because
the resolver will by default try to serialize your content as JSON, and obviously fail unless it is
valid JSON.
You can also return stream objects using for instance the [return-value] slot, at which point
ASP.NET Core will automatically stream your content back over the response object, and Dispose
your stream automatically for you afterwards. This allows you to for instance return large files back
to the client without loading them into memory first. If you do this, you'll have to change
your Content-Type accordingly.
Cookies
Since cookies have more parameters than just a simple key/value declaration, the [response.cookies.set]
slot takes the following arguments.
[value] - The string content of your cookie
[expires] - Absolute expiration date of your cookie, as a Hyperlambda date value
[http-only] - Boolean value declaring whether or not the cookie should only be accessible on the server
[secure] - Boolean value declaring whether or not cookie should only be transmitted from the client to the server over a secure (https) connection
[domain] - Domain value of your cookie
[path] - Path value of your cookie
[same-site] - Same-site value of your cookie
Only the [value] from above is mandatory. To delete a cookie on the client, set the expiration date to a value
in the past.
Project website
The source code for this repository can be found at github.com/polterguy/magic.endpoint, and you can provide feedback, provide bug reports, etc at the same place.
