3

I have a number of dynamic pages in my eXist-db application which until this moment have been tested through hardcoded inputs. Now I'm at the point where I have to venture into a domain I'm the least comfortable with: URL mapping/rewriting in eXist-db.

Here is the fundamental scenario. When a user wants to look at a medieval document on my site, they can either search a list and click a link or input a URL directly. Regardless, the url would be:

www.foo.com/doc/MS609-0020

Where /doc/ says you are looking at documents, and MS609-0020 is actually the name of a document (which reflects the XML file ie MS609-0020.xml).

All such documents will output through a generic page located at:

/db/apps/deheresi/document.html

This document.html contains templates that need the document name MS609-0020.xml from the request in order to know what to process. I've named this globally available variable $currentdoc. Using this variable, different templates get the data needed to assemble the document.html.

Then the HTML is sent to the browser under www.foo.com/doc/MS609-0020

If the document name is not found, the user is directed to a 'document not found page'.

As you can see, my explanation is basic - reflecting the fact I've still got training wheels on with eXist and XQuery.

I've read through the chapters in Siegel/Retter's eXist book, but how remapping is done in modules, and passed with the required parameters...is not coming together for me no matter how much I hack through this.

I would be immensely appreciative for any guidance, from pseudo code and instructions, to some snippets. Once I see how this one request can be handled, then it will come together for me as a practice.

Many thanks in advance.

NB: I should add that this will be a public site with no permissions required to view the documents.

NB 2: I'll be throwing a bounty on this question once eligible.

Below, diagram from Siegel/Retter's book.

Diagram from Siegel/Retter

jbrehr
  • 775
  • 6
  • 19
  • First, an important consideration/question. On the server where this app will live, is eXist running behind a reverse proxy (which could take requests for "/" and forward them to eXist as requests for "/exist/apps/my-app" - the collection with your XQuery modules and HTML templates)? Or is eXist exposed directly to the internet (in which case you'd need to change eXist's URL structure to eliminate "/exist" and forward requests for "/" to "/db/apps/my-app"). While the latter is possible, the former is recommended. (See https://exist-db.org/exist/apps/doc/production_good_practice.) – Joe Wicentowski Oct 24 '18 at 13:09
  • To be honest, I'm not responsible for having installed the server. I have to work through a server admin to go beyond the exist-db environment. I've been provided with a setup where http://medieval-inquisition.huma-num.fr/exist/apps/deheresi/index.html points to `db/apps/heresi` (I've been given credentials to access `db/apps/deheresi` ). That of course is an unwieldy URL, which I prefer the public see simple as http://medieval-inquisition.huma-num.fr/index , etc. – jbrehr Oct 24 '18 at 13:16
  • I've added the bounty to this question. – jbrehr Oct 26 '18 at 11:57
  • It is difficult to answer your question without seeing what you have tried, and what errors you encountered along the way. Many things can interfere with url rewriting, that have nothing to do with exit. You will need to wiggle it down to a MWE including your document.html, xquery module, controller-config, etc – duncdrum Oct 27 '18 at 09:46
  • It's actually more basic than that. I'm looking for a simple explanation of the order of operations for someone who has virtually no experience with the practice of URL mapping and rewriting. Something like: "when eXist receives request to serve a page, 'this' happens 'here'. If you want eXist to parse the request to find and load a document, you create 'x' in module 'y' to push 'z' information 'here', etc." ...on a fresh install of eXist. The eXist book leaves out this basic description as it assumes the user has already done some form of URL mapping and rewriting. – jbrehr Oct 27 '18 at 10:15
  • @jbrehr Please confirm that the assumptions in the introduction to my answer are acceptable, and I'll proceed. Otherwise, I'll adjust. – Joe Wicentowski Oct 27 '18 at 14:32
  • @joewiz This is fantastic. And yes, I'll use eXist's `controller.xql`. – jbrehr Oct 27 '18 at 16:11
  • @joewiz I should clarify: in describing URL rewriting I hope you'll also cover how to serve up the requested page (the 'mapping' part?). Thanks again. – jbrehr Oct 27 '18 at 17:17
  • @jbrehr Ok, I've finished the answer. Let me know if anything needs further clarification. – Joe Wicentowski Oct 28 '18 at 03:56
  • The URL to "production_good_practice" article in my first comment above was somehow mangled. And stackoverflow won't let me edit the old comment. So here's the correct URL: https://exist-db.org/exist/apps/doc/production_good_practice.xml. – Joe Wicentowski Oct 28 '18 at 04:02

1 Answers1

3

This answer, which covers only basic URL rewriting, assumes that the eXist application is built using what I'll call "basic" eXist app architecture:

  • The app, which we'll call "my-app", is stored in /db/apps/my-app.
  • No modifications have been made to the Jetty configuration files in EXIST_HOME/tools/jetty/etc, the XQuery servlet configuration file in EXIST_HOME/webapp/WEB-INF/web.xml, or the base configuration for eXist's URL rewriting at webapp/WEB-INF/controller-config.xml.
  • This means the application can be accessed at http://localhost:8080/exist/apps/my-app. (If the goal is to serve this application via a URL like http://my-server/ and have this routed to eXist at http://localhost:8080/exist/apps/my-app, this is best handled by a reverse proxy server, which is beyond the scope of "basic" eXist app architecture.)
  • We will handle URL rewriting via eXist's URL rewriting facility - i.e., writing a controller.xql file - rather than via RESTXQ.

With these assumptions, we can create completely custom URLs, allowing us to take a URL like:

  • http://localhost:8080/exist/apps/my-app/doc.xq?filename=my-document.xml

and make this same resource available via:

  • http://localhost:8080/exist/apps/my-app/doc/my-document

To accomplish this, we need to create a new XQuery main module, named controller.xql (it must be called this exactly, and we'll call it the app's "controller") in the root collection of your app: /db/apps/my-app/controller.xql. This is a special module, which eXist looks for first when a request comes for a path in the /apps URL space (e.g., http://localhost:8080/exist/apps/...). While typically an app will have only one controller, eXist supports multiple controllers; eXist looks in the target collection and then up the collection tree, from deepest branch up to the root /db/apps collection.

The purpose of the controller is to take information about the request - the path requested and other information about the app's context - and return a special kind of directive, which tells eXist how to route the request. The key information about the request is exposed to your query in a series of external variables (variables you don't have to define that eXist sets for you and that you can reference), including, most importantly, $exist:path - the portion of the request URL that comes after the path to the collection containing the controller. So in the URL above, $exist:path would equal /doc/my-document.

Now, let's create the directive that takes this path and forwards this request (formulated using with the filename parameter to your actual endpoint:

xquery version "3.1";

declare variable $exist:path external;
declare variable $exist:resource external;
declare variable $exist:controller external;
declare variable $exist:prefix external;
declare variable $exist:root external;

if (starts-with($exist:path, "/doc/")) then
    <dispatch xmlns="http://exist.sourceforge.net/NS/exist">
        <forward url="{$exist:controller}/doc.xq">
            <add-parameter name="filename" value="{$exist:resource}.xml"/>
        </forward>
    </dispatch>

else
    <ignore xmlns="http://exist.sourceforge.net/NS/exist"/>

In the first block of code (the prolog), this controller explicitly declares the variables that eXist provides to the controller. In the main portion of the query, the conditional expression handles two cases:

  1. If the path requested begins with /doc/, then we want the request to be handled by doc.xq - an XQuery module in our app at /db/apps/my-app/doc.xq. Rather than type this full path, we can replace /db/apps/my-app with $exist:controller - which is the database path to the collection where the active controller is found. We also know that doc.xq requires a filename parameter, so we explicitly construct that from the $exist:resource variable, which always provides us with the portion of the requested URL after the final slash - e.g., my-document in our example URL above. From the perspective of the query that the controller forwards the request to (e.g., doc.xq), it thinks it has received the request directly, and it can access request parameters via the request:get-parameter() function. For example, doc.xq could simply contain:

    xquery version "3.1";
    
    let $doc := request:get-parameter("filename", ())
    return
        <p>Who's looking for {$filename}?</p>
    

    And the request for the URL above would return <p>Who's looking for my-document.xml?</p>.

  2. For any other requests, we will let these through without performing any forwarding or other URL action.

If you are using eXist's HTML templating facility, you will likely be forwarding requests to a template (document.html) instead of a query (doc.xq). In this case, the directive gets a little more complicated:

    <dispatch xmlns="http://exist.sourceforge.net/NS/exist">
        <forward url="{$exist:controller}/document.html"/>
        <view>
            <forward url="{$exist:controller}/modules/view.xql">
                <add-parameter name="filename" value="{$exist:resource}.xml"/>
            </forward>
        </view>
    </dispatch>

Here, we are forwarding the request to document.html, but then the result is passed through eXist's templating handler module, which is conventionally stored in your app's modules/view.xql file. (Note that the <add-parameter> directive has moved to the 2nd <forward> directive.)

The other external variables (namely, $exist:root and $exist:prefix) and other kinds of directives (namely, <redirect> and other sub-elements) are described in the eXist documentation's page on URL Rewriting, http://exist-db.org/exist/apps/doc/urlrewrite. While the page currently warns that it is out of date, I think it is still a good resource.

The strength of eXist's URL Rewriting facility is that you can use the full expressiveness of XQuery to determine how your application routes and receives URLs. The weakness is that the controller can become a long chain of conditionals that is hard to maintain; just try to keep the controller logic as simple as possible. It can also take some time to learn the external variables; I'd suggest making ample use of logging (via the util:log or console:log functions to output the values of the external variables and other information about the request, as available via request:get-url() and the other request:get-* functions) to see what's going on with each request, until you get the hand of it. You may also find it instructive to survey controller files in other apps whose source code is available, and ask more questions here!

Joe Wicentowski
  • 5,159
  • 16
  • 26
  • This is really instructive and helpful, thank you. Question: eXist creates a default `controller.xql`, which come populated with a set of `if/else if' statements for processing various requests. I assume the `if` statement you created above gets inserted among those existing statements (or replaces one, in the case of a conflict)? Also: is there a reason to use nested `if` instead of `switch` or `type switch`)? – jbrehr Oct 28 '18 at 07:43
  • 1
    My answer assumed you were starting from scratch, but if you’re working with an existing app or an app you generated, then yes, you’ll need to figure out where to insert your new rule or customize it. This can be tricky, no doubt. Good question about switch vs. if. Switch is fine if you have a list of predefined strings that you’re expecting. But for more sophisticated logic like starts-with, contains, or matches, you need if. – Joe Wicentowski Oct 28 '18 at 13:23
  • 1
    I am tripping over implementing this as your example uses `view`as part of a path, but when eXist does a default creation of an app, it creates `view.xq` by default with extant code. So when you are passing an actual request with `view` in the path, I'm not sure whether you are using eXist's default `view.xq` with `view.html`. So, I'm not sure if you mean to say in your example of using eXist's HTML template, whether one should pass the '' as eXist's default, or you mean I should be re-writing `view.xql` specifically to handle my requests... – jbrehr Oct 30 '18 at 12:57
  • 1
    I can see how that's confusing! Thank you for pointing that out. I have just revised the answer to use better names. Now, in the first example, we take a request for `/doc/my-document` and forward it to `/db/apps/my-app/doc.xq?filename=my-document.xml`. And in the second example, we take the same request and forward it instead to `/db/apps/my-app/document.html`, which is then passed through (*view*ed through) `/db/apps/my-app/modules/view.xql`. – Joe Wicentowski Oct 30 '18 at 13:23
  • Thanks for updating the answer, that clarifies things. Can I set a global variable instead of passing a parameter? My `document.html` calls 5 other templates that each get the same parameter but I'd rather just set my global variable `$globalvar:currentdoc` to the value of the incoming document request. But it seems global variables can't be dynamically updated in XQuery? If not, if I set the `add-parameter` to my document identifier, will that pass through to the subsequent templates called by `document.html`? Thanks again ever so much....I'm just about there! :) – jbrehr Oct 30 '18 at 14:23
  • I'm avoiding model, and just trying to pass a parameter. I've posted the resulting error as a new question: https://stackoverflow.com/questions/53067680/exist-db-passing-parameters-in-url-mapping-through-controller-xql It seems the parameter is not being received by the templates... – jbrehr Oct 30 '18 at 15:25
  • My mistake; thank you again for raising the question. The `` directive has to move to the 2nd `` so `modules/view.xql` has access to the parameters. I've updated the answer above. – Joe Wicentowski Oct 30 '18 at 18:46