gerben
/
web-annotation-discovery-server
Store and publish annotations on the web, as described in the Web Annotation Discovery proposal.
Code Issues 0 Pull Requests 0 Releases 0 Activity
Browse Source

Initial publication

main
Gerben 2 years ago
commit
9afea92555
31 changed files with 4096 additions and 0 deletions
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +61
    -0
      .gitignore
  2. +8
    -0
      .reuse/dep5
  3. +9
    -0
      LICENSES/MIT.txt
  4. +10
    -0
      LICENSES/Unlicense.txt
  5. +87
    -0
      Readme.md
  6. +49
    -0
      app.ts
  7. +89
    -0
      bin/www.ts
  8. +3
    -0
      config/config.json
  9. +3
    -0
      config/users.json
  10. +2615
    -0
      package-lock.json
  11. +31
    -0
      package.json
  12. +41
    -0
      public/images/icon.svg
  13. +68
    -0
      public/stylesheets/style.css
  14. +12
    -0
      routes/db.ts
  15. +194
    -0
      routes/handlers/annotation.ts
  16. +146
    -0
      routes/handlers/collection.ts
  17. +37
    -0
      routes/handlers/user.ts
  18. +218
    -0
      routes/ldp.ts
  19. +59
    -0
      routes/render/renderAnnotation.ts
  20. +22
    -0
      routes/render/renderCollection.ts
  21. +13
    -0
      routes/render/renderUser.ts
  22. +101
    -0
      routes/router.ts
  23. +30
    -0
      routes/util.ts
  24. +13
    -0
      tsconfig.json
  25. +7
    -0
      views/annotation.hbs
  26. +37
    -0
      views/collection.hbs
  27. +3
    -0
      views/error.hbs
  28. +18
    -0
      views/index.hbs
  29. +15
    -0
      views/layouts/main.hbs
  30. +47
    -0
      views/partials/annotation.hbs
  31. +50
    -0
      views/user.hbs

+ 61
- 0
.gitignore View File

@@ -0,0 +1,61 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*

# Runtime data
pids
*.pid
*.seed
*.pid.lock

# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov

# Coverage directory used by tools like istanbul
coverage

# nyc test coverage
.nyc_output

# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt

# Bower dependency directory (https://bower.io/)
bower_components

# node-waf configuration
.lock-wscript

# Compiled binary addons (https://nodejs.org/api/addons.html)
build/Release

# Dependency directories
node_modules/
jspm_packages/

# Typescript v1 declaration files
typings/

# Optional npm cache directory
.npm

# Optional eslint cache
.eslintcache

# Optional REPL history
.node_repl_history

# Output of 'npm pack'
*.tgz

# Yarn Integrity file
.yarn-integrity

# dotenv environment variables file
.env

# next.js build output
.next

+ 8
- 0
.reuse/dep5 View File

@@ -0,0 +1,8 @@
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: Gerben
Upstream-Contact: <>
Source: https://code.treora.com/gerben/web-annotation-discovery-server

Files: *
Copyright: 2022 Gerben
License: Unlicense

+ 9
- 0
LICENSES/MIT.txt View File

@@ -0,0 +1,9 @@
MIT License

Copyright (c) <year> <copyright holders>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

+ 10
- 0
LICENSES/Unlicense.txt View File

@@ -0,0 +1,10 @@
This is free and unencumbered software released into the public domain.

Anyone is free to copy, modify, publish, use, compile, sell, or distribute this software, either in source code form or as a compiled binary, for any purpose, commercial or non-commercial, and by any means.

In jurisdictions that recognize copyright laws, the author or authors of this software dedicate any and all copyright interest in the software to the public domain. We make this dedication for the benefit of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of relinquishment in perpetuity of all present and future rights to this software under copyright law.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

For more information, please refer to <http://unlicense.org/>

+ 87
- 0
Readme.md View File

@@ -0,0 +1,87 @@
# Web Annotation Discovery Server

This server enables people to store and publish annotations on the web, as described in the [Web Annotation Discovery][] proposal.

It has been developed and tested in combination with the [Web Annotation Discovery WebExtension][] browser extension. Other clients that speak the [Web Annotation Protocol][] should be compatible too (if, for writing to the server, they support [HTTP Basic Authentication][]).

[Web Annotation Discovery]: https://code.treora.com/gerben/web-annotation-discovery
[Web Annotation Discovery WebExtension]: https://code.treora.com/gerben/web-annotation-discovery-webextension
[Web Annotation Protocol]: https://www.w3.org/TR/annotation-protocol/
[HTTP Basic Authentication]: https://datatracker.ietf.org/doc/html/rfc7617


## Status

Minimum viable product, or primarily a proof-of-concept: This software demonstrates a mechanism for web annotation in practice, enabling people to try it out and hopefully move forward with the idea.

The software can be used for collaborative annotation, but be prepared for possible technical glitches (including data loss or leakage), mediocre user experience, and awful interface design.


## Try it out

### Install & run

Tested with [NodeJS][] version 16 (LTS).

1. Clone this repository: `git clone …`

2. Install dependencies: `npm install`

3. Add a user (or several) in `config/users.json`

4. Run the server: `npm start`

### Use

Point your browser to your server (by default it’s `http://localhost:8080/`), click your username and create a collection. Name it e.g. “My notes”. The browser should prompt you now for your username and password, which it remembers for subsequent requests. (Without any obvious way to log out, however..)

For the next steps, ensure you have the [Web Annotation Discovery WebExtension][] installed and enabled in your browser.

Visit the collection’s URL, e.g. `http://localhost:8080/alice/my_notes/`. The browser extension will discover the `<link>` tag in the collection page, and offer to subscribe to this collection.

The browser extension will then display notes in the collection on pages you visit, and let you select this collection for any new annotations you make. See the documentation of the browser extension for more explanation on its usage.

[NodeJS]: https://nodejs.org/
[Web Annotation Discovery WebExtension]: https://code.treora.com/gerben/web-annotation-discovery-webextension

### Security & Access control

When publicly accesible, always run this server behind a reverse proxy that only permits connection via TLS. Authentication is performed using the HTTP Basic Authentication, so passwords would otherwise be sent around the world in plaintext.

The list of known users and their passwords is configured in `config/users.json`. The default is `alice` with password `secret123`.

By default, data is stored in `/tmp/annotationdb`; configure this in `config/config.json`.

Note that all annotations on the server are publicly readable. Only the owner of a collection can create/modify annotations in it. More granular control has not (yet) been implemented, but your reverse proxy could shield of a user’s whole path (i.e. `/username/*`) or individual collections (`/username/collection_name/*`) if desired.


## Develop

Install dependencies using `npm install`; then running `npm start` should spin it up. Or `npm run dev` runs it with live reloading upon source file changes.

### Code tour

This server is derived from [simple-annotation-server][] by Jan Kaßel, converted from JavaScript and [hapi][] to [TypeScript][] and [Express][], among other changes; but still using the [Level][] database. Besides the JSON interface for the [Web Annotation Protocol][], this version also renders users, collections and annotations as HTML.

For authentication (for `POST`/`PUT`/`DELETE` requests), HTTP Basic Authentication is used, as it is simple and (somewhat) supported by most web browsers.

[simple-annotation-server]: https://github.com/jankaszel/simple-annotation-server
[hapi]: https://hapi.dev/
[TypeScript]: https://typescriptlang.org/
[Express]: https://expressjs.com/
[Level]: https://github.com/Level/level
[Web Annotation Protocol]: https://www.w3.org/TR/annotation-protocol/


## Licence

Much of this code is derived from [simple-annotation-server][] by Jan Kaßel, MIT-licensed.

Anything else in this repository is free and unencumbered software released into the public domain.

[simple-annotation-server]: https://github.com/jankaszel/simple-annotation-server


## Acknowledgements

This project was [funded](https://nlnet.nl/project/WebAnnotation/) through the NGI0 Discovery Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme.

+ 49
- 0
app.ts View File

@@ -0,0 +1,49 @@
import express from 'express';
import createError from 'http-errors';
import annotationsRouter from './routes/router.js';
import { create } from 'express-handlebars';
import { annotationHbsParams } from './routes/render/renderAnnotation.js';
import { fileURLToPath } from 'url';

var app = express();

// view engine setup
app.set('views', fullPath('views'));
const hbs = create({
extname: 'hbs',
helpers: {
annotationHbsParams(arg) {
return annotationHbsParams(arg, false);
},
},
});
app.engine('hbs', hbs.engine);
app.set('view engine', 'hbs');

app.use(express.json({ type: ['application/json', 'application/ld+json'] }));
app.use(express.urlencoded({ extended: false }));
app.use(express.static(fullPath('public')));

app.use('/', annotationsRouter);

// catch 404 and forward to error handler
app.use(function (req, res, next) {
next(createError(404));
});

// error handler
app.use(function (err, req, res, next) {
// set locals, only providing error in development
res.locals.message = err.message;
res.locals.error = req.app.get('env') === 'development' ? err : {};

// render the error page
res.status(err.status || 500);
res.render('error');
});

export default app;

function fullPath(filename: string): string {
return fileURLToPath(new URL(filename, import.meta.url));
}

+ 89
- 0
bin/www.ts View File

@@ -0,0 +1,89 @@
#!/usr/bin/env node

/**
* Module dependencies.
*/

import app from '../app.js'
import http from 'http'

/**
* Get port from environment and store in Express.
*/

var port = normalizePort(process.env.PORT || '8080');
app.set('port', port);

/**
* Create HTTP server.
*/

var server = http.createServer(app);

/**
* Listen on provided port, on all network interfaces.
*/

server.listen(port);
server.on('error', onError);
server.on('listening', onListening);

/**
* Normalize a port into a number, string, or false.
*/

function normalizePort(val: string) {
var port = parseInt(val, 10);

if (isNaN(port)) {
// named pipe
return val;
}

if (port >= 0) {
// port number
return port;
}

return false;
}

/**
* Event listener for HTTP server "error" event.
*/

function onError(error: any) {
if (error.syscall !== 'listen') {
throw error;
}

var bind = typeof port === 'string'
? 'Pipe ' + port
: 'Port ' + port;

// handle specific listen errors with friendly messages
switch (error.code) {
case 'EACCES':
console.error(bind + ' requires elevated privileges');
process.exit(1);
break;
case 'EADDRINUSE':
console.error(bind + ' is already in use');
process.exit(1);
break;
default:
throw error;
}
}

/**
* Event listener for HTTP server "listening" event.
*/

function onListening() {
var addr = server.address();
var bind = typeof addr === 'string'
? 'pipe ' + addr
: 'port ' + addr?.port;
console.log('Listening on %s', bind)
}

+ 3
- 0
config/config.json View File

@@ -0,0 +1,3 @@
{
"databasePath": "/tmp/annotationdb"
}

+ 3
- 0
config/users.json View File

@@ -0,0 +1,3 @@
{
"alice": "secret123"
}

+ 2615
- 0
package-lock.json
File diff suppressed because it is too large
View File


+ 31
- 0
package.json View File

@@ -0,0 +1,31 @@
{
"name": "server",
"version": "0.0.0",
"private": true,
"type": "module",
"scripts": {
"start": "tsx ./bin/www.ts",
"dev": "tsx watch ./bin/www.ts"
},
"dependencies": {
"@types/etag": "^1.8.1",
"@types/express": "^4.17.14",
"@types/http-errors": "^1.8.2",
"@types/node": "^18.7.21",
"@types/uuid": "^8.3.4",
"escape-string-regexp": "^4.0.0",
"etag": "^1.8.1",
"express": "~4.16.1",
"express-basic-auth": "^1.2.1",
"express-handlebars": "^6.0.6",
"http-errors": "~1.6.3",
"level": "^8.0.0",
"uuid": "^9.0.0",
"validate-web-annotation": "^0.3.0",
"web-annotation-utils": "git+https://code.treora.com/gerben/web-annotation-utils#latest"
},
"devDependencies": {
"tsx": "^3.9.0",
"typescript": "^4.8.3"
}
}

+ 41
- 0
public/images/icon.svg View File

@@ -0,0 +1,41 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
width="10mm"
height="10mm"
viewBox="0 0 10 10"
version="1.1"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:cc="http://creativecommons.org/ns#">
<rect
style="display:inline;fill:#ffff00;stroke:#000000;stroke-width:1.39897"
width="10.617116"
height="8.6010437"
x="2.9194891"
y="0.69947922"
ry="2.0371485"
rx="2.0371485"
clip-path="none" />
<path
d="M 11.327955,8.1663486 H 10.41402 q -0.153787,0 -0.250453,-0.074451 -0.09667,-0.07883 -0.145,-0.1926961 L 9.5440248,6.6072619 H 6.9120694 L 6.4375265,7.8992017 q -0.035151,0.1007275 -0.1362114,0.1839371 -0.10106,0.08321 -0.2504531,0.08321 H 5.1281396 L 7.6282776,1.8336543 H 8.8322105 Z M 7.2152495,5.7751652 H 9.2408446 L 8.4675154,3.6686466 Q 8.4103946,3.528504 8.3488798,3.3401874 8.287365,3.1474912 8.2258501,2.9241389 8.1643353,3.1474912 8.1028205,3.3401874 8.0456995,3.5328834 7.9885787,3.6774055 Z"
style="font-weight:bold;font-size:8.77336px;line-height:1.25;font-family:Lato;stroke-width:0.427093" />
<metadata>
<rdf:RDF>
<cc:License
rdf:about="http://creativecommons.org/publicdomain/zero/1.0/">
<cc:permits
rdf:resource="http://creativecommons.org/ns#Reproduction" />
<cc:permits
rdf:resource="http://creativecommons.org/ns#Distribution" />
<cc:permits
rdf:resource="http://creativecommons.org/ns#DerivativeWorks" />
</cc:License>
<cc:Work
rdf:about="">
<cc:license
rdf:resource="http://creativecommons.org/publicdomain/zero/1.0/" />
</cc:Work>
</rdf:RDF>
</metadata>
</svg>

+ 68
- 0
public/stylesheets/style.css View File

@@ -0,0 +1,68 @@
body {
font-size: 18px;
font-family: 'Lato', sans-serif;
}

body > main {
margin: 3em auto;
max-width: 30em;
}

a {
color: unset;
}

a:hover,
a:focus {
color: darkblue;
}

.annotationBody {
border: #999 1px solid;
border-radius: 10px;
box-shadow: 10px 10px 10px #9996;
margin: 10px 0px;
padding: 10px 40px 10px 20px;
background-color: lightyellow;
}

.targets {
list-style: none;
}

.textquote {
padding: 0 1em;
margin-top: 0.4em;
}
.textquote mark {
background: #ff99;
}
.textquote::before {
content: open-quote;
}
.textquote::after {
content: close-quote;
}

.small {
font-size: smaller;
}

.large {
font-size: larger;
}

a.plainlink {
text-decoration: none;
color: unset;
}

.infobox {
margin: 3em 0;
padding: 1em;
background: #f8f8f8;
}

details > summary {
cursor: pointer;
}

+ 12
- 0
routes/db.ts View File

@@ -0,0 +1,12 @@
// Copyright (c) 2020 Jan Kaßel
// Copyright (c) 2022 Gerben
//
// SPDX-License-Identifier: MIT

import { Level } from 'level';
import { databasePath } from '../config/config.json';

const db = new Level<string, any>(databasePath, {
valueEncoding: 'json',
});
export default db;

+ 194
- 0
routes/handlers/annotation.ts View File

@@ -0,0 +1,194 @@
// Copyright (c) 2020 Jan Kaßel
// Copyright (c) 2022 Gerben
//
// SPDX-License-Identifier: MIT

import { v4 as uuid } from 'uuid';
// import validateAnnotation from 'validate-web-annotation';
import db from '../db.js';
import { Container, sendAnnotation, extractAnnotationIdFromUrl } from '../ldp.js';
import type { Request, Response } from 'express';
import { asArray } from 'web-annotation-utils';

export async function createAnnotation(req: Request, res: Response) {
const collectionKey = `${req.params.user}/${req.params.collection}`;
try {
await db.get(collectionKey);
} catch (err: any) {
if (err.notFound) {
res.status(404).send('Not found');
} else {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
}
return;
}
if (!req.body) {
res.status(400).send('Bad request: Missing request body');
return;
}

const annotation = req.body;
// TODO Improve validator; it appears to disagree about the date format.
// if (!validateAnnotation(annotation, { optionalId: true })) {
// res.status(400).send('Bad request: Invalid body schema');
// return;
// }

const id = uuid();
const annotationKey = `${collectionKey}/${id}`;
if (annotation.id) {
annotation.via = [...asArray(annotation.via), annotation.id];
}
annotation.id = id;

try {
await db.get(annotationKey);
res.status(409).send('Conflict');
} catch (err: any) {
if (!err.notFound) {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
return;
}

annotation.creator ??= {
type: 'Person',
nickname: req.params.user,
};

await db.put(annotationKey, annotation);

const collectionInfo = await db.get(collectionKey);
const containerInfo = new Container(req, collectionKey, collectionInfo);
res
.status(201)
.header('Location', `/${annotationKey}`)
.header('Content-Location', `/${annotationKey}`);
sendAnnotation(req, res, annotation, containerInfo);
}
}

export async function getAnnotation(req: Request, res: Response) {
const collectionKey = `${req.params.user}/${req.params.collection}`;

try {
await db.get(collectionKey);
} catch (err: any) {
if (err.notFound) {
res.status(404).send('Not found');
} else {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
}
return;
}

const annotationKey = `${collectionKey}/${req.params.annotation}`;
try {
const annotation = await db.get(annotationKey);
const collectionInfo = await db.get(collectionKey);
const containerInfo = new Container(req, collectionKey, collectionInfo);
sendAnnotation(req, res, annotation, containerInfo);
} catch (err: any) {
if (err.notFound) {
res.status(404).send('Not found');
} else {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
}
return;
}
}

export async function updateAnnotation(req: Request, res: Response) {
const collectionKey = `${req.params.user}/${req.params.collection}`;
try {
await db.get(collectionKey);
} catch (err: any) {
if (err.notFound) {
res.status(404).send('Not found');
} else {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
}
return;
}

const annotation = req.body;
if (
!annotation ||
// TODO Improve validator; it appears to disagree about the date format.
// || !validateAnnotation(annotation)
!annotation.id
) {
res.status(400).send('Bad request: Invalid body schema');
return;
}

const collectionInfo = await db.get(collectionKey);
const containerInfo = new Container(req, collectionKey, collectionInfo);
const normalizedId = extractAnnotationIdFromUrl(
annotation.id,
containerInfo.url,
);
if (!normalizedId) {
res
.status(400)
.send(
`Bad Request: Annotation ID did not match the expected container IRI: ${containerInfo.url}`,
);
return;
}
const contractedAnnotation = {
...annotation,
id: normalizedId,
};
const annotationKey = `${collectionKey}/${contractedAnnotation.id}`;
try {
await db.get(annotationKey);
await db.put(annotationKey, contractedAnnotation);

const collectionInfo = await db.get(collectionKey);
const containerInfo = new Container(req, collectionKey, collectionInfo);
sendAnnotation(req, res, contractedAnnotation, containerInfo);
} catch (err: any) {
if (err.notFound) {
res.status(404).send('Not found');
} else {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
}
return;
}
}

export async function deleteAnnotation(req: Request, res: Response) {
const collectionKey = `${req.params.user}/${req.params.collection}`;
try {
await db.get(collectionKey);
} catch (err: any) {
if (err.notFound) {
res.status(404).send('Not found');
} else {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
}
return;
}

const annotationKey = `${collectionKey}/${req.params.annotation}`;
try {
await db.get(annotationKey);
await db.del(annotationKey);
res.status(204).send();
} catch (err: any) {
if (err.notFound) {
res.status(404).send('Not found');
} else {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
}
return;
}
}

+ 146
- 0
routes/handlers/collection.ts View File

@@ -0,0 +1,146 @@
// Copyright (c) 2020 Jan Kaßel
// Copyright (c) 2022 Gerben
//
// SPDX-License-Identifier: MIT

import db from '../db.js';
import {
Container,
PagedContainer,
sendPage,
sendContainer,
expandAnnotation,
} from '../ldp.js';
import { getPrefixedEntries } from '../util.js';
import type { Request, Response } from 'express';

export interface CollectionInfo {
name: string,
label: string,
}

export async function createCollection(req: Request, res: Response) {
if (!req.body?.name) {
res.status(400).send('Bad request');
return;
}

const { name, label } = req.body;
const collectionKey = `${req.params.user}/${name}`;
try {
await db.get(collectionKey);
res.status(409).send('Conflict');
return;
} catch (err: any) {
if (!err.notFound) {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
return;
}

const collection: CollectionInfo = {
name,
label,
};
await db.put(collectionKey, collection);

res.format({
html: () => {
res.redirect(303, `${req.baseUrl}/${collectionKey}/`);
},
default: () => {
res
.status(201)
.header('Location', `${req.baseUrl}/${collectionKey}/`)
.header('Content-Location', `${req.baseUrl}/${collectionKey}/`);
req.params.collection = name;
getCollection(req, res);
}
});
}
}

export async function getCollection(req: Request, res: Response) {
const collectionKey = `${req.params.user}/${req.params.collection}`;
let collectionInfo: CollectionInfo;
try {
collectionInfo = await db.get(collectionKey);
} catch (err: any) {
if (err.notFound) {
res.status(404).send('Not found');
} else {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
}
return;
}

const pageNumber = req.query.page
? Number.parseInt(req.query.page as string)
: null;
const iris = req.query.iris === '1';
const containerInfo = new Container(req, collectionKey, collectionInfo);

try {
const annotations = (
await getPrefixedEntries(db, `${collectionKey}/`, true)
).map(([_, value]) => value);

// Sort most recent first (based on annotations’ self-reported modified/created date).
annotations.sort((a, b) =>
(a.modified ?? a.created ?? '') < (b.modified ?? b.created ?? '')
? 1
: -1,
);
const collection = new PagedContainer(
req,
collectionKey,
collectionInfo,
annotations.map((annotation) =>
expandAnnotation(annotation, containerInfo),
),
);

if (pageNumber !== null) {
sendPage(res, collection, pageNumber, iris);
} else {
// Embed the first page unless the client prefers otherwise.
const embedFirstPage = req.headers.prefer?.includes(
'http://www.w3.org/ns/ldp#PreferMinimalContainer',
)
? false
: true;
sendContainer(req, res, collection, iris, embedFirstPage);
}
} catch (err: any) {
if (err.notFound) {
res.status(404).send('Not found');
} else {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
}
}
}

export async function deleteCollection(req: Request, res: Response) {
const collectionKey = `${req.params.user}/${req.params.collection}`;
try {
await db.get(collectionKey);

const entries = await getPrefixedEntries(db, `${collectionKey}/`);
for (const [key, _] of entries) {
await db.del(key);
}

await db.del(collectionKey);
res.status(204).send();
} catch (err: any) {
if (err.notFound) {
res.status(404).send('Not found');
} else {
console.error(req.method, req.path, err);
res.status(500).send('Internal server error');
}
return;
}
}

+ 37
- 0
routes/handlers/user.ts View File

@@ -0,0 +1,37 @@
// Copyright (c) 2020 Jan Kaßel
// Copyright (c) 2022 Gerben
//
// SPDX-License-Identifier: MIT

import type { Request, Response } from 'express';
import db from '../db.js';
import { renderUser } from '../render/renderUser.js';
import { getPrefixedEntries } from '../util.js';
import users from '../../config/users.json';
import { CollectionInfo } from './collection.js';

export async function getUser(req: Request, res: Response) {
const name = req.params.user;
if (!(name in users)) {
res.status(404).send('Not found');
return;
}
try {
const collections: CollectionInfo[] = (await getPrefixedEntries(db, `${name}/`, true)).map(
([_, value]) => value,
);

const user = {
name,
collections,
};

res.format({
html: () => renderUser(req, res, user),
default: () => res.send(user),
});
} catch (err: any) {
console.error(req.method, req.path, err);
res.status(500).send('Internal Server Error');
}
}

+ 218
- 0
routes/ldp.ts View File

@@ -0,0 +1,218 @@
// Copyright (c) 2020 Jan Kaßel
// Copyright (c) 2022 Gerben
//
// SPDX-License-Identifier: MIT

import escapeString from 'escape-string-regexp';
import etag from 'etag';
import type { Request, Response } from 'express';
import { CollectionInfo } from './handlers/collection.js';
import { renderAnnotation } from './render/renderAnnotation.js';
import { renderCollection } from './render/renderCollection.js';

export function extractAnnotationIdFromUrl(
fullAnnotationUrl: string,
containerUrl: string,
) {
const escapedUrl = escapeString(containerUrl);
const pattern = new RegExp(`^${escapedUrl}\/([0-9a-z-]+)$`);
const matches = fullAnnotationUrl.match(pattern);
return !matches ? null : matches[1];
}

export function expandAnnotation(annotation, containerInfo: Container) {
return {
...annotation,
id: `${containerInfo.url}/${annotation.id}`,
};
}

export function contractAnnotation(annotation, containerInfo: Container) {
return {
...annotation,
id: extractAnnotationIdFromUrl(annotation.id, containerInfo.url),
};
}

export class Container {
url: string;
name: string;
label: string;

constructor(
private req: Request,
public containerPath: string,
collectionInfo: CollectionInfo,
) {
const host = this.req.headers['x-forwarded-host'] || this.req.headers.host;
this.url = `${this.req.protocol}://${host}${this.req.baseUrl}/${this.containerPath}`;
this.name = collectionInfo.name;
this.label = collectionInfo.label;
}
}

export class PagedContainer extends Container {
public total: number;
public pageSize: number;
private _getPage: (pageNumber: number, pageSize: number) => any[];

constructor(
req: Request,
containerPath: string,
collectionInfo: CollectionInfo,
getPage: any[],
);
constructor(
req: Request,
containerPath: string,
collectionInfo: CollectionInfo,
getPage: (pageNumber: number, pageSize: number) => any[],
total: number,
pageSize: number,
);
constructor(
req: Request,
containerPath: string,
collectionInfo: CollectionInfo,
getPage: ((pageNumber: number, pageSize: number) => any[]) | any[],
total?: number,
pageSize?: number,
) {
super(req, containerPath, collectionInfo);

if (Array.isArray(getPage)) {
const items = getPage;
this._getPage = () => items;
this.total = items.length;
this.pageSize = Infinity;
} else {
this._getPage = getPage;
this.total = total as number;
this.pageSize = pageSize as number;
}
}

getPage(pageNumber: number) {
return this._getPage(pageNumber, this.pageSize);
}

get lastPage(): number {
return this.pageSize === Infinity
? 0
: Math.floor(this.total / this.pageSize);
}
}

export function sendPage(
res: Response,
container: PagedContainer,
pageNumber: number,
iris: boolean,
) {
if (pageNumber > container.lastPage) {
res.status(404).send('Not found');
return;
}

const items = container.getPage(pageNumber);
const page = {
'@context': 'http://www.w3.org/ns/anno.jsonld',
id: `${container.url}/?page=${pageNumber}&iris=${iris ? 1 : 0}`,
type: 'AnnotationPage',
partOf: {
id: `${container.url}/?iris=${iris ? 1 : 0}`,
total: container.total,
},
startIndex: pageNumber === 0 ? 0 : container.pageSize * pageNumber,
items: iris ? items.map((item) => item.id) : items,
};

res.type('application/ld+json; profile="http://www.w3.org/ns/anno.jsonld"');
res.header('Allow', 'HEAD, GET, OPTIONS');

res.send(page);
}

export function sendContainer(
req: Request,
res: Response,
container: PagedContainer,
iris: boolean,
embedFirstPage?: boolean,
) {
let firstPage: any;
if (embedFirstPage) {
const items = container.getPage(0);
firstPage = {
id: `${container.url}/?page=0&iris=${iris ? 1 : 0}`,
type: 'AnnotationPage',
startIndex: 0,
items: iris ? items.map((item) => item.id) : items,
};
} else {
firstPage = `${container.url}/?iris=${iris ? 1 : 0}&page=0`;
}

const collectionJsonldObj = {
'@context': [
'http://www.w3.org/ns/anno.jsonld',
'http://www.w3.org/ns/ldp.jsonld',
],
id: `${container.url}/?iris=${iris ? 1 : 0}`,
type: ['BasicContainer', 'AnnotationCollection'],
total: container.total,
label: container.label,
first: firstPage,
...(container.lastPage > 0 && {
last: `${container.url}/?iris=${iris ? 1 : 0}&page=${container.lastPage}`,
}),
};

res.header('ETag', etag(JSON.stringify(collectionJsonldObj)));

res.format({
html: () =>
renderCollection(req, res, {
collection: collectionJsonldObj,
container,
user: req.params.user,
}),
default: () => {
res.header('Link', [
'<http://www.w3.org/ns/ldp#BasicContainer>; rel="type"',
'<http://www.w3.org/TR/annotation-protocol/>; rel="http://www.w3.org/ns/ldp#constrainedBy"',
]);
res.header(
'Accept-Post',
'application/ld+json; profile="http://www.w3.org/ns/anno.jsonld"',
);
res.type(
'application/ld+json; profile="http://www.w3.org/ns/anno.jsonld"',
);
res.header('Allow', 'HEAD, GET, POST, OPTIONS');

res.send(collectionJsonldObj);
},
});
}

export function sendAnnotation(
req: Request,
res: Response,
annotation,
containerInfo: Container,
) {
annotation = expandAnnotation(annotation, containerInfo);
res.header('ETag', etag(JSON.stringify(annotation)));
res.format({
html: () => renderAnnotation(req, res, { annotation, containerInfo }),
default: () => {
res.header('Allow', 'OPTIONS,HEAD,GET,PUT,DELETE');
res.header('Link', '<http://www.w3.org/ns/ldp#Resource>; rel="type"');
res.type(
'application/ld+json; profile="http://www.w3.org/ns/anno.jsonld"',
);
res.send(annotation);
},
});
}

+ 59
- 0
routes/render/renderAnnotation.ts View File

@@ -0,0 +1,59 @@
import type { Request, Response } from 'express';
import { getTargetQuotes, getTargetUrls } from 'web-annotation-utils';
import { escapeHtml } from '../util.js';

export function renderAnnotation(
req: Request,
res: Response,
{ annotation, ...params },
) {
res.render('annotation', {
title: `Annotation`,
...annotationHbsParams(annotation, true),
...params,
});
}

export function annotationHbsParams(annotation, verbose?: boolean) {
const targetUrls = getTargetUrls(annotation.target);
return {
verbose,
annotation,
json: JSON.stringify(annotation, null, 2),
created: annotation.created
? new Date(annotation.created).toDateString()
: '<i>?</i>',
modified:
annotation.modified && new Date(annotation.modified).toDateString(),
bodyHtml: annotation.bodyValue ?? annotationBodyToHtml(annotation.body),
targetUrl: targetUrls.length === 1 ? targetUrls[0] : undefined,
targetUrls: targetUrls.length > 1 ? targetUrls : undefined,
targetQuotes: getTargetQuotes(annotation.target),
};
}

function annotationBodyToHtml(body) {
const iframe = (bodyUrl: string) =>
`<iframe sandbox="" src=${bodyUrl}></iframe>`;
try {
if (body === undefined || body === null) {
return '';
}
if (Array.isArray(body)) {
return body.map(annotationBodyToHtml).join('\n<br/>\n');
}
if (typeof body === 'string') {
return iframe(body);
}
if (body.type === 'Choice') {
return annotationBodyToHtml(body.items[0]);
}
if (body.type === 'TextualBody') {
if (body.format === 'text/html')
return `<iframe sandbox="" csp="default-src: data: unsafe-inline;" srcdoc="${body.value}"></iframe>`;
else return `<p>${escapeHtml(body.value)}</p>`;
}
} catch (err: any) {
return '<i>Error while rendering annotation body.</i>';
}
}

+ 22
- 0
routes/render/renderCollection.ts View File

@@ -0,0 +1,22 @@
import type { Request, Response } from 'express';
import { escapeHtml } from '../util.js';

export function renderCollection(req: Request, res: Response, { collection, container, user }) {
const nicestName = collection.label || container.name || '';
res.render('collection', {
title: `Annotation collection “${nicestName}” of ${user}`,
collection,
nicestName,
user,
head: `
<link
rel="alternate"
type='application/ld+json; profile="http://www.w3.org/ns/anno.jsonld"'
href=""
title="${escapeHtml(nicestName)}"
/>
`,
json: JSON.stringify(collection, null, 2),
annotations: collection.first.items,
});
}

+ 13
- 0
routes/render/renderUser.ts View File

@@ -0,0 +1,13 @@
import type { Request, Response } from 'express';

export function renderUser(
req: Request,
res: Response,
{ name, ...other },
) {
res.render('user', {
title: `Annotations of ${name}`,
name,
...other,
});
}

+ 101
- 0
routes/router.ts View File

@@ -0,0 +1,101 @@
// Copyright (c) 2020 Jan Kaßel
// Copyright (c) 2022 Gerben
//
// SPDX-License-Identifier: MIT

import express, { NextFunction, Request, Response } from 'express';
import basicAuth from 'express-basic-auth';
import users from '../config/users.json';
import {
createCollection,
getCollection,
deleteCollection,
} from './handlers/collection.js';
import {
createAnnotation,
getAnnotation,
updateAnnotation,
deleteAnnotation,
} from './handlers/annotation.js';
import { getUser } from './handlers/user.js';

var router = express.Router();

const authHandler = basicAuth({
users,
challenge: true,
});

// Require authentication only for write methods
router.use((req, res, next) => {
if (['POST', 'PUT', 'DELETE'].includes(req.method.toUpperCase()))
authHandler(req, res, next);
else next();
});

declare global {
namespace Express {
interface Request {
auth?: {
user: string;
password: string;
};
}
}
}

function checkIfAuthorised(req: Request, res: Response, next: NextFunction) {
if (req.auth?.user === req.params.user) {
next();
} else {
res.status(403).send('Forbidden');
}
}

function ensureTrailingSlash(req: Request, res: Response, next: NextFunction) {
const [originalPath, query] = req.originalUrl.split('?');
if (originalPath.endsWith('/')) {
next();
} else {
const newPath = `${originalPath}/${query ? `?${query}` : ''}`;
res.redirect(301, newPath);
}
}

router.get('/', ensureTrailingSlash, (req, res, next) => {
res.render('index', {
title: `Annonation server`,
users: Object.keys(users),
});
});
router.get('/login', authHandler, (req, res, next) => {
if (req.auth?.user) {
res.redirect(`${req.baseUrl}/${req.auth?.user}`);
} else {
res.status(500).send('Something wrong2.');
}
});
router.get('/logout', (req, res, next) => {
basicAuth({
users: {},
challenge: true,
})(req, res, next);
});
router.get('/:user/', ensureTrailingSlash, getUser);
router.post('/:user/', checkIfAuthorised, createCollection);
router.get('/:user/:collection/', ensureTrailingSlash, getCollection);
router.delete('/:user/:collection/', checkIfAuthorised, deleteCollection);
router.post('/:user/:collection/', checkIfAuthorised, createAnnotation);
router.get('/:user/:collection/:annotation', getAnnotation);
router.put(
'/:user/:collection/:annotation',
checkIfAuthorised,
updateAnnotation,
);
router.delete(
'/:user/:collection/:annotation',
checkIfAuthorised,
deleteAnnotation,
);

export default router;

+ 30
- 0
routes/util.ts View File

@@ -0,0 +1,30 @@
// Copyright (c) 2020 Jan Kaßel
// Copyright (c) 2022 Gerben
//
// SPDX-License-Identifier: MIT

import { randomBytes } from 'crypto';
import type { Level } from 'level';

export function generateKey(length = 16) {
return randomBytes(length).toString('hex');
}

export async function getPrefixedEntries(db: Level<string, any>, prefix: string, shallow?: boolean) {
const entries: [string, any][] = [];
for await (const entry of db.iterator({ gt: prefix })) {
if (!entry[0].startsWith(prefix)) break;
if (shallow && entry[0].slice(prefix.length).includes('/')) continue;
entries.push(entry);
}
return entries;
}

export function escapeHtml(s: string) {
return s
.replace('<', '&lt;')
.replace('>', '&gt;')
.replace('&', '&amp;')
.replace('"', '&quot;')
.replace("'", '&apos;');
}

+ 13
- 0
tsconfig.json View File

@@ -0,0 +1,13 @@
{
"compilerOptions": {
"target": "es6",
"module": "NodeNext",
"rootDir": "./",
"outDir": "./build",
"esModuleInterop": true,
"allowSyntheticDefaultImports": true,
"strict": true,
"noImplicitAny": false,
"resolveJsonModule": true
}
}

+ 7
- 0
views/annotation.hbs View File

@@ -0,0 +1,7 @@
<p class="small" style="margin-bottom: 3em;">
<a class="plainlink" href="{{containerInfo.url}}">← Back to collection</a>
</p>

<h1>Annotation</h1>

{{>annotation}}

+ 37
- 0
views/collection.hbs View File

@@ -0,0 +1,37 @@
<p class="small" style="margin-bottom: 3em;">
<a class="plainlink" href="..">← Back to all collections of {{user}}</a>
</p>

<h1>Annotation collection “{{nicestName}}”</h1>

<p>This collection contains {{collection.total}} annotations.</p>

<details class="infobox">
<summary>
How to get these annotations in your browser?
</summary>

<h2>Subscribe this collection</h2>
<p>With an annotation-capable browser (extension), you can view these annotations while you visit the pages they target.</p>
<p>A collection is an annotation ‘feed’: Your browser should give you the option to import/subscribe to this collection, and then update it periodically.</p>

<h2>Creating annotations</h2>
<p>
If this is your own collection, you can add and edit annotations from your annotation-capable browser.
</p>
<p>
First, ‘subscribe’ to this collection in your browser. You can then choose to use this collection for storing any new annotations you make.
</p>
</details>

<h2>Annotations in this collection</h2>
{{#each annotations}}
<div style="margin-bottom: 3em;">
{{>annotation (annotationHbsParams this)}}
</div>
{{/each}}

<details>
<summary style="margin-top: 4em;" class='small'>View this collection as JSON</summary>
<pre>{{json}}</pre>
</details>

+ 3
- 0
views/error.hbs View File

@@ -0,0 +1,3 @@
<h1>{{message}}</h1>
<h2>{{error.status}}</h2>
<pre>{{error.stack}}</pre>

+ 18
- 0
views/index.hbs View File

@@ -0,0 +1,18 @@
<h1>Annotation server</h1>

<p>
This website hosts <a href="https://www.w3.org/annotation/">web annotations</a>. The users listed below host their public annotations here. You can subscribe to their collections of annotations to see them on the pages they target.
</p>
<p>
To view annotations on web pages, you need a web browser (extension) that supports web annotations.
</p>
<p class="annotationBody">
Note this software is experimental. Do not rely on it for serious business.
</p>

<h2>Users</h2>

<p>This annotation server is home to the following users:</p>
{{#each users}}
<p class='large'><a href='{{this}}/'>{{this}}</a></p>
{{/each}}

+ 15
- 0
views/layouts/main.hbs View File

@@ -0,0 +1,15 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<title>{{title}}</title>
<link rel="icon" href="/images/icon.svg" type="image/svg+xml">
<link rel='stylesheet' href='/stylesheets/style.css' />
{{~#if head}}{{{head}}}{{/if}}
</head>
<body>
<main>
{{{body}}}
</main>
</body>
</html>

+ 47
- 0
views/partials/annotation.hbs View File

@@ -0,0 +1,47 @@
{{#if verbose}}
<div class='annotationBody'>
{{{bodyHtml}}}
</div>
{{else}}
<a href='{{annotation.id}}' class='plainlink'>
<div class='annotationBody'>
{{{bodyHtml}}}
</div>
</a>
{{/if}}

<p style='margin-bottom: 0;'>{{#if verbose}}This is an annotation {{/if}}on:
{{#if targetUrl}}<a href='{{targetUrl}}'>{{targetUrl}}</a>{{/if}}
</p>
{{#if targetUrls}}
<ul class='targets'>
{{#each targetUrls}}
<li><a href='{{this}}'>{{this}}</a></li>
{{/each}}
</ul>
{{/if}}

{{#if targetQuotes}}
{{#if verbose}}<p>On the text:</p>{{/if}}
{{#each targetQuotes}}
<blockquote class='textquote'><mark>{{this}}</mark></blockquote>
{{/each}}
{{/if}}

{{#if verbose}}
<p class='small'>Created on {{{created}}}</p>
{{#if modified}}
<p class='small'>Last modified {{{modified}}}</p>
{{/if}}

<details style='margin-top: 4em;'>
<summary class='small'>View this annotation as JSON</summary>
<pre>{{json}}</pre>
</details>

<script
type='application/ld+json; profile="http://www.w3.org/ns/anno.jsonld"'
>
{{{json}}}
</script>
{{/if}}

+ 50
- 0
views/user.hbs View File

@@ -0,0 +1,50 @@
<p class='small' style='margin-bottom: 3em;'>
<a class='plainlink' href='..'>← Back to all users</a>
</p>

<h1>Annotation collections of {{name}}</h1>

<details class="infobox">
<summary>
How to use/create an annotation collection
</summary>

<h2>Subscribe to a collection</h2>
<p>With an annotation-capable browser (extension), you can view annotations from the collections below while you visit the pages they target.</p>
<p>Each collection is an annotation ‘feed’: Choose a collection below, and your browser should give you the option to import/subscribe to this collection, and then update it periodically.</p>

<h2>Create a collection</h2>
<p>
If you are {{name}}, you can add and edit annotations from your annotation-capable browser.
</p>
<p>
Create a collection below. Keep your password at hand (perhaps your browser has remembered it for you).
</p>

<form method="POST">
<div>
<input required name='label' placeholder='Name of collection' oninput="generateSlug(event)" style="font-size: 1em;"/>
<button style="font-size: large;">Create</button>
</div>
<p style="margin-top: .2em;">
/{{name}}/<input required name='name' placeholder='' style="color: grey; background: none; border: none; font-size: 1em;" />
</p>
</form>
<script>
function generateSlug(event) {
const slug = event.target.value.toLowerCase().replaceAll(/\s+/g, '_');
event.target.form.elements.name.value = slug;
}
</script>

<h2>Start creating annotations</h2>
<p>
Open a collection, then ‘subscribe’ to it in your annotation-enabled browser. You can then choose to create new annotations in this collection.
</p>
</details>

{{#each collections}}
<p>
<a href='{{name}}/'>{{#if label}}{{label}}{{else}}{{name}}{{/if}}</a>
</p>
{{/each}}

Write Preview
Loading…
Cancel
Save