Introduction

Cette documentation est récente et en cours d’édition. Elle n’est donc pas complète et/ou peut être inexacte.

Qu’est ce que NumaHOP ?

NumaHOP est une solution logicielle pour la gestion de la chaine de numérisation. Cela comprends l’import de méta données, le constat d’état des documents, jusqu’à la diffusion en masse. Elle sert intermédiaire entre les prestataires de numérisation, les établissement et les plateformes d’exports. Ce processus est semi-automatisé avec des contrôles manuels.

Ce que NumaHOP peut faire

NumaHOP est composé de module pouvant:

• importer des notices bibliographiques et les convertir en DublinCore.
• visionner et valider les images et les notices des documents numérisés.
• rédiger des constats d’états pour les documents envoyés a un prestataire de numérisation.
• utiliser des workflow pour gérer le processus de numérisation.
• exporter les documents vers les services d’archivages et les bibliothèques numériques.
• produire des fichier dérivés comme de l’OCR, des fichier METS, des images de différentes résolutions.

NumaHOP peut diffuser les documents en masse vers les plateformes suivantes:

• un serveur SFTP.
• une instance OMEKA/OMEKA-S.
• le CINES.
• Internet Archive.

NumaHOP peut aussi être partagé par plusieurs établissement pour réaliser des projets de numérisation conjoints.

Installation.

Obtenir NumaHOP

On peut obtenir NumaHOP par trois méthodes:

• Depuis les pages de publication de version. (conseillé)
• Compiler depuis le code source.
• Dockerised solution. See the Docker Usage chapter.

Compiler depuis le code source

Programmes nécessaires

• git
• une distribution de la jdk (java developpement kit) version 17.
• just (optionnel)
• maven > 3.6.0 (optionnel)

Before the installation process, clone the git repository and cd into it:

git clone https://github.com/NumaHOP/NumaHOP.git numahop && cd numahop

If you don’t have maven installed you can run this command:

chmod +x mvnw

And use ./mvnw instead of mvn.

Then you can run the compilation command for numahop:

mvn package 

# Or using the just file
just d

Si vous rencontrez des erreurs pendant la compilation lancez mvn -Dfast=true package. Cela desactive toute les execution non necessaires pour la compilation comme les tests, l’analyse du code ou la vérification du formatage du code. Une fois que la compilation est faite un fichier numahop-yy.mm.jar devrait être présent dans le dossier target. Pour l’installation deplacez ce fichier vers /opt/pgcn/numahop.jar. C’est le chemin d’installation par default de NumaHOP.

Dépendances à l’exécution

Now use your package manager to install the necessary runtime dependencies for numahop:

• mariadb > 10.*
• elasticsearch > 8.*
• open-jdk-17
• tesseract-ocr-all
• libimage-exiftool-perl
• imagemagick
• libyaz4j (optionnel)

The libyaz4j dependency is only used by the Z39.50 search queries. If you don’t need the Z39.50 queries you can skip this dependency.

Base de données

Maintenant nous allons créer une base de données vide pour NumaHOP. NumaHOP crée les tables automatiquement donc vous n’avez pas besoin de les créer vous même. Voir la page sur liquibase pour plus d’informations.

mariadb

Lancez ces commandes en remplaçant les bouts de texte a substituer <db-name>, <db-user-name> et <db-user-pwd>. Notez ceux-ci vous allez en avoir besoin plus tard.

MariaDB [(none)]>

La commande pour créer la base:

CREATE DATABASE <db-name>;

Puis créez l’utilisateur que NumaHOP utilisera pour se connecter a la base de données.

CREATE USER '<db-user-name>'@localhost IDENTIFIED BY <db-user-pwd>;
GRANT CREATE, ALTER, DROP, INSERT, SELECT, UPDATE, INDEX, DELETE ON '<db-name>'.* TO '<db-user-name>'@localhost;
FLUSH PRIVILEGES;

Remarque: Les droits donnés a l’utilisateur sont les droits minimaux pour que NumaHOP puisse éditer la base de données.

Vous pouvez vérifier que tous est bon en lançant les commandes SHOW DATABASES; et SHOW GRANTS FOR '<db-user-name>'@localhost.

Elastic Search

Installez Elastic Search en suivant leur documentation (lien. Et assurez vous que l’instance Elastic Search soit accessible par localhost ou une URL.

A faire: Recommander une configuration pour l’utiliser avec NumaHOP.

Le serveur mail (optionnel mais important)

NumaHOP n’as pas besoin du serveur de mail pour tourner cependant certaines fonctionnalités comme le changement de mot de passe ne seront pas fonctionnelles.

Configuration de NumaHOP

A présent nous pouvons configurer NumaHOP en suivant le guide de configuration. Après cette étape vous devriez avoir une instance NumaHOP fonctionnelle.

Si vous voulez gérer le processus de NumaHOP avec systemd

Creez le fichier /etc/systemd/system/numahop.service:

[Unit]
Description=NumaHOP
After=syslog.target

[Service]
ExecStart=/opt/pgcn/numahop.jar --spring.profiles.active=prod # for activating certain spring profiles see the config chapter for more information.
SuccessExitStatus=143
StandardOutput=null
User=numahop 

[Install]
WantedBy=multi-user.target

Puis lancez les commandes (selon vos droits utilisateur elle peuvent demander d’être lancer avec l’utilisateur root):

systemctl daemon-reload # After adding/modifying the file
systemctl start numahop # For starting numahop
systemctl stop numahop # For stoping numahop

Si vous n’avez pas les droits vous pouvez creez le fichier ~/.config/systemd/user/numahop.service contenant:

[Unit]
Description=NumaHOP
After=syslog.target

[Service]
ExecStart=/opt/pgcn/numahop.jar --spring.profiles.active=prod
SuccessExitStatus=143
StandardOutput=null

[Install]
WantedBy=multi-user.target

Et utiliser les commandes:

systemctl --user daemon-reload # After adding/modifying the file
systemctl --user start numahop # For starting numahop
systemctl --user stop numahop # For stoping numahop

Si tous a fonctionne correctement vous pouvez acceder a l’interface graphique en vous connectant au serveur. Vous pouvez vous diriger vers le guide de post installation.

NumaHOP Configuration.

La configuration de NumaHOP se fait dans des fichiers yaml nomes application.ymlou application-<profile>.yml.

Quand NumaHOP demare il cherce and charge les fichiers de configuration dans cet ordre:

• Dans les ressources a l’interieur du packet jar.
• Dans le dossier ./config (relatif a son repertoir d’execution).

Profiles

NumaHOP peut charger plusieurs fichier de configuration a l’aide d’un system de profile.En premier il charge le fichier application.yml et ensuite les fichiers des profils actifs.Chaque chargement écrase les clès de configuration déjà définies.

To activate a profile the flag --spring.profiles.active can be passed to the war:

./numahop.jar --spring.profile.active=dev,prod

The resulting files will be sourced in this order:

• À l’interieur du jar:
  • application.yml
  • application-dev.yml
  • application-prod.yml
• À l’interieur du répertoir de travail:
  • ./config/application.yml
  • ./config/application-dev.yml
  • ./config/application-prod.yml

application.yml

La plupart des propriétées disponible dans les fichier de configuration sontdéfinies par SpringBoot certaines le sont par NumaHOP. Toute les utilisation despropriétées de NumaHOP peuvent être trouvées en utilisant la commande: grep -e "@Value\(\$\{.*\}\)"

Propriétées importantes et utiles

Instances (Doit être définie)

Chaque etablissements dans numahop doit etre declare ici. L’id doit correspondreavec l’id de l’etablissement dans la table library en base de donnees.

instance:
    libraries: lib1, lib2, lib3

Admin (Doit être définie)

Le compte admin est le seul compte present apres installation.

admin:
    login: # default: admin
    password: # Important to override. default: hash of "password"

Le mot de pass est encrypte en utilisant bcrypt. Vous pouvez obtenir un bonmot de passe avec cette commande:

head /dev/urandom | tr -dc "a-zA-Z0-9"| fold -w 20 | head -n 1

Ensuit encrypez le avec une implementation de bcrypt. Point d’attention toutes lesimplémentations de bcrypt ne sont pas sécurisées. Une option est d’encrypter le mot depasse en utilisant python3 avec le packet python3-bcrypt:

echo "$pwd" | python3 -c <<PYTHON
import bcrypt, getpass;
print(bcrypt.hashpw(input().encode(), bcrypt.gensalt()).decode())'
PYTHON

Ou pwd est votre mot de passe.

Jhipster Remember Me key (Doit être définie)

C’est la cle qui permet au service rememberme de fonctionner. Elle doit être uniquepour votre installation.

jhipster.security.rememberme.key: # No defaults must be defined.

De la meme maniere que pour le mot de passe vous pouvez obtenirune bonne clé comme ceci:

head /dev/urandom | tr -dc "a-zA-Z0-9"| fold -w 20 | head -n 1

Base de données et elasticsearch (Doit être définie)

Les informations dont NumaHOP a besoin pour se connecter à la base de données et elasticsearch.

spring:
    datasource:
        url: # The url to access the mariadb data base often: jdbc:mariadb://localhost:3306/<db-name>
        username: # The database user
        password: # The database password
    elasticsearch:
        uris:
            - <uri> # default: http://localhost:9200
            # ...

Sytème de fichier

Ces options de configuration définissent où NumaHOP peut trouver les executables dont il a besoin, ainsi que l’endroit où il doit placer les différent fichiers qu’il génère.

base-path: #default /opt/pgcn

imageMagick:
    convert: # default : /usr/bin/convert
    identify: # default : /usr/bin/identify

exifTool:
    process: # default : /usr/bin/exiftool

tesseract:
    process: # default : /usr/bin/tesseract

storage:
    binaries: # default : ${base-path}/DAM/DAM_Repository

nativeLibraries:
    path: # default : /opt/svcmgmt/lib

messaging:
    template:
        path: # default : ${base-path}/Messaging/MessagingTemplate


uploadPath:
    condition_report: # default : ${base-path}/upload/condition_report
    ead: # default : ${base-path}/upload/ead
    import: # default : ${base-path}/upload/import
    library: # default : ${base-path}/upload/library
    user: # default : ${base-path}/upload/user
    template: # default : ${base-path}/upload/templates

export:
    ssh:
        knownHosts: # default : ${base-path}/config/known_hosts

services:
    cines:
        aip: # default : ${base-path}/cines/aip
        cache: # default : ${base-path}/cines/cache
        xsd:
            sip: # default : ${base-path}/xsd/sip.xsd
    archive:
        alto: # default : ${base-path}/archive/alto
        text: # default : ${base-path}/archive/text
    metaDatas:
        path: # default : ${base-path}/metadatas
    deliveryreporting:
        path: # default : ${base-path}/deliveryReporting
    omeka:
        cache: # default : ${base-path}/omeka/cache
    ftpexport:
        cache: # default : ${base-path}/ftpexport/cache
    digitalLibraryDiffusion:
        cache: # default : ${base-path}/digitalLibraryDiffusion/cache

Serveur

Le port utilisé par NumaHOP.

server:
    port: # default: 80

Tailles d’images

images:
    format:
        default:
            thumbHeight: # default: 150 
            thumbWidth: # default: 150
            viewHeight: # default: 1417
            viewWidth: # default: 982
            printHeight: # default: 2835
            printWidth: # default: 1964

Cron jobs

Plusieurs services de NumaHOP sont annotés avec @Scheduled. Ce sontdes con jobs configurable.

cron:
    removeOldTokens: # default: 0 0 0 * * ?
    cleanLocks: # default: 0 30 0 * * *
    rebuildIndex: # Rebuild elastic search index. default: 0 40 0 * * *
    cleanDeliveryFiles: # default: 0 0 1 * * ?
    cinesExport: # Try to export eligible document units to the CINES. default: 0 30 1 * * ?
    localExport: # Try to export eligible document units via SFTP. default: 0 0 4 * * ?
    cleanTemporaryFiles: # Clean up temporary files for each library. default: 0 0 2 * * ?
    internetArchiveExport: # Try to export eligible document units to Internet Archive. default: 0 26 17 * * ?
    gestDeliveredFiles: # Save or delete the archived documents. default: 0 00 18 * * SAT
    cinesUpdateStatus: # Download status of CINES export from the mailbox. default: 0 10 * * * *
    omekaExport: # Try to export eligible document units to an Omeka instance. default: 0 0 3 * * ?
    cleanExportCaches: # default: 0 30 6 * * ?
    docUnitUpdateArk: # Retrieve Internet archive urls of archived documents. default: 0 0 6 * * ?
    digitalLibraryExport: # default: 0 0 3 * * ?

La valeur de chaque clé est une expression cron utilisée pour configurer quand la tache doit être executée.

Mail

spring:
    mail:
        host: # default: localhost
        port: # default: 25
        user:
        password:
        from: # default: contact-numahop@numahop.fr
        tls: # default: false
        auth: # default: false
        activated: # default: false

Post installation

Now you have an installed and working numahop instance. But the base installation of numahop is rather empty. The first thing you must create before anything else is the libraries declared inside the yaml configuration files.

Here is a list of things you might want to create:

• User Groups and Users
• Workflow Models
• Contidion Report
• Mail Templates
• Export Configurations
• Bibliographic Records mappings
• Automatic Controls settings

There is a default account with login admin. This account should only be used to create new users and the administration tools.

Adding a Library

There should already be a library present with identifier lib_library. This one can be used. If you want more or one with another identifer it should be configured right after installation. Each library must be referenced in the database and its identifier registered in the config file.

For each libraries declared create a library through the interface. Then you need to edit the libraries inside the database to make the column identifier match the names declared inside the yaml files. If the library you created isn’t defined in the yaml file you need to edit the yaml file and add an identifer to it and restart numahop.

-- Print all the libraries you declared through the interface.
SELECT name FROM lib_library\G
-- Modify the identifier 
-- <lib-identifier> corresponds to the identifier set inside the yaml file.
-- <lib-name> corresponds to the name given to the library at the creation in the web interface.
UPDATE lib_library SET identifier='<lib-identifier>' WHERE name='<lib-name>';
-- Note it is ok to filter by the name as NumaHOP normally enforces it to be unique. 

Once the libraries are created you can create the rest of the objects you need for your use of NumaHOP.

If you have created other objects before the library column update by accident here is how to fix it:

-- Check and note what is the identifier generated by numahop by default.
-- <old-uuid> is what it is referenced as from now on.
SELECT identifier from lib_library WHERE name='<lib-name>';

-- We need to disable the foreign key contraints checks while modifying the libraries otherwise 
-- the contraints will prevent the modification because other tables reference this identifier.
SET FOREIGN_KEY_CHECKS = 0;
UPDATE lib_library SET identifier='<lib-identifier>' WHERE name='<lib-name>';
SET FOREIGN_KEY_CHECKS = 1;

-- Print all tables corresponding to created objects that have a library field and have rows.
SELECT c.TABLE_NAME from information_schema.columns c JOIN information_schema.tables t
ON c.TABLE_NAME = t.TABLE_NAME WHERE c.COLUMN_NAME = 'library' AND t.TABLE_ROWS >=1;
-- For each table corresponding to the created object fix the foreign key.
-- <old-uuid> is the value we 
UPDATE <talble> SET library='<lib-identifier>' WHERE library='<old-uuid>';

First user and user groups

I advise you create a admin user profile with all rights allowed for each libraries as the admin account configured in the yaml file can do some of the things but there are a lot of functionality that are disallowed on this account.

You can also create user profiles that suits your needs.

WorkflowModel and workflow groups.

Mappings

FtpConfiguration

Dockerisation de NumaHOP

La plupart des commandes

git clone https://github.com/NumaHOP/NumaHOP.git numahop && cd numahop

It is recommended to use at least docker v28.4.0.

Docker and composition.

All the docker configuration and files live in the src/main/docker directory.

The Docker image of NumaHOP is built in 2 stages first a base is established. In this base we install the runtime dependencies of NumaHOP and setup the system user that numahop will use.

Then a run stage is built. We setup the working directory for numahop and the yaz library. This step is only needed to be run once as the rebuild after changes in the source code of NumaHOP will reuse the latest run image as a base.

This run stage will then be used by the jib maven plugin to install the numahop build artifact inside the image. The new image is named numahop and will have the tag latest.

The composition is declared with a compose file per service with toplevel compose files that serve as aliases.

Here is the list of services:

• numahop-db: the mariadb container.
• numahop-es: the elasticsearch container.
• numahop-mail: the mail container.
• numahop-nh: the numahop application container.

The two file aliases are all and env. env regroups the db, es, and mail containers all adds to these the application container.

Build the docker image.

Using the raw commands:

docker build -t numahop-run src/main/docker --target run 
mvn clean compile -Pdocker,webapp 

Using the justfile:

just setup && just build docker

After this you should have two image present in your local docker repository:

docker image ls

Managing the docker composition.

Before using the composition ensure you have built the docker image as described precedently.

To start up the docker composition:

# Note: all docker compose commands with the -f argument should be started from the root of the NumaHOP source code repository
docker compose -p numahop -f src/main/docker/docker-compose.all.yml up
# Or using just
just docker up

You now should be able to access your NumaHOP instance at http://localhost:8080. You also can consult numahop logs with:

docker compose -p numahop -f src/main/docker/docker-compose.all.yml logs 
# Or using just
just docker logs

To stop all the services:

docker compose -p numahop -f src/main/docker/docker-compose.all.yml stop

just docker stop # stop les services
just docker down # stop les services et supprime les conteneurs
just docker clean # stop les services et supprime les conteneurs en detachant les volumes

If you want to execute a command inside a container:

docker compose -p numahop exec -it [service-name] -- [command]
# For example to connect to the db use
docker compose -p numahop exec -it numahop-db mariadb -Dnumahop -unumahop -pnumahop
# Open a file
docker compose -p numahop exec -it numahop-db mariadb -Dnumahop -unumahop -pnumahop

For developpements it is usefull to be able to reset the data of your composition. The persisted data in docker is stored inside volumes:

docker volume ls

All the volumes used by the composition are in the form numahop_numahop-[image]-volume.

These volumes can be detached/deleted from the composition:

# Pour toute les images
docker compose -p numahop -f src/main/docker/docker-compose.[image].yml rm -f -s -v
docker volume rm numahop_numahop-[image]-volume

# Or using the justfile 
just docker clean
just docker reset nh
just docker reset es
just docker reset db 

Architecture de NumaHOP

Schéma d’Entrées Sorties

Documentations externes utiles

• angularjs
• springboot

Code organization and structure.

Back-End

The root of the back end code is located at src/main/java/fr/progilone/pgcn.

The logic is split up in two ways. Firstly the logic is split vertically in the stack by repositories, services and controllers (in ascending order) these are stored inside folders in the root:

• repositories: Abstraction of storage methods. Can be stored inside the database or the file system.
• services: Core of the business logic.
• web: REST API containing the controllers. Makes the interface to the front-end.

Then the logic is split horizontally by modules operating on a specific primitives. Most of them are pretty straight forward. All the modules are:

• es: ElasticSearch module.
• user: users and their abilities (rights). The authentication is not handled there.
• train:
• lot:
• library:
• *configuration: Modules managing dynamic configurations (configuration that can be edited at runtime).
• workflow:
• delivery:
• document:
• statistics:
• exchange: archiving and diffusion of the document units.
• sample: Module managing the sample mode of digitalized documents. (No reason of being a separated module refactor needed).
• check: Module managing automatic checks (Not triggered or performed by users).
• help: Module managing the help page content.
• imagemetadata:
• multilotdelivery:

There are also other folders in the root which store various configurations as java classes:

• security: The configuration of spring security to allow login and
• config: All configuration classes.

The last folder in the root is the domain folder containing the data models organized by modules representing the different object stored in the database. We can also find additional classes in the folders dto and jaxb. The ones in dto are suffixed with DTO which are objects sent by the Controller (here a REST Controller) to the View (the front-end) and those contained in jaxb are pre-generated classes from schema definitions of the metadata formats used.

Front-End

The front-end root is located at src/main/webapp.

The majority of the functionality is in the folder script.

The DocUnit Module

This module manages document units.

Module files

Domain Objects:

Services:

Controllers:

Manages document units.

The Workflow Module.

In this module the primitive we operate on are workflows. In NumaHOP workflows are tracking the life-cycle of document units.

A workflow is constituted of states. Each states is either processed automatically or require some user interaction in order to be processed. Some states are process automatically and don’t require much user interaction such as the Diffusion, Archiving, Automatic Checks, or Delivery Waiting.

Once the condition required for the state to validate the workflow completes it and start the next state. Some states can also be processed in parallel such as diffusion and archiving states.

The workflow is configured with a workflow model. The workflow model to use on a document unit is configured either on the project, the lot, the train. It can also be defined on the document unit itself. If a workflow model is not present it will inherit the model of the parent document unit grouping. (project -> lot -> train -> document unit)

Order of states

• DOCUMENT_INITIALISATION
• SLIPS_GENERATION
• VALIDATION_CONDITION_REPORT
• VALIDATION_SLIPS_CONDITION_REPORT
• CONDITION_REPORT_BEFORE_DIGITIZATION
• WAITING_FOR_DIGITIZATION
• CONDITION_REPORT_AFTER_DIGITIZATION
• WAITING_FOR_DELIVERY
• WAITING_FOR_AUTOMATIC_CHECKS
• WAITING_FOR_QUALITY_CONTROL
• DOCUMENT_PREREJECTION
• DOCUMENT_PREVALIDATION
• DOCUMENT_VALIDATION
• BIBLIOGRAPHIC_RECORDS_VALIDATION
• CONTROL_REPPORTS
• DOCUMENT_ARCHIVING
• DOCUMENT_BROADCAST
• DOCUMENT_BROADCAST_OMEKA
• DOCUMENT_BROADCAST_DIGITAL_LIBRARY
• DOCUMENT_BROADCAST_LOCAL
• CLOSING_DOCUMENT

Special state as it is manually inserted in the workflow in case of a rejection of the state DOCUMENT_PREREJECTION:

• WAITING_FOR_REDELIVERY

Each states holds a status:

• NOT_STARTED
• PENDING
• FINISHED
• CANCELED
• FAILED
• TO_WAIT
• WAITING
• WAITING_NEXT_COMPLETED
• TO_SKIP
• SKIPPED

Module files Domain objects: Domain Ojects:

• WorkflowModel
• WorkflowGroup
• WorkflowStateModel
• WorkflowStateKey
• WorkflowStateStatus
• DocUnitWorkflow
• DocUnitState (abstract class).
• All states classes implementation.

Services:

• WorkflowModelService
• WorkflowModelStateService
• WorkflowService
• DocUnitWorkflowService
• WorkflowGroupService
• WorkflowGroupValidationService
• UiWorkflowService
• UiWorkflowGroupService
• UiWorkflowModelService

Controllers:

• WorkflowController
• WorkflowGroupController
• WorkflowModelController

Workflow are created on a document unit when the workflows are started on the lot level. But once a workflow started its steps can’t be modified. When creating a workflow we use a workflow model. Workflow models are a template of a workflow. Each state has different statuses when

API auto documentation.

Note: The open-api auto documentation is incorect because NumaHOP doesn’t respect the open-api specification causing some handlers to be merged into one in the documentation. This will be fixed when the api is redesigned see #8.

• Open Api documentation
• Scraped documentation

File storage in NumaHOP.

NumaHOP stores a good amount of its files on the filesystem.

Here is a list of the services that store files:

• MetaDatasCheckService
• DeliveryReportingService
• ConditionReportAttachmentService
• ImportReportService
• CinesRequestHandlerService
• ExportCinesService
• ExportSipService
• ExportEadService
• TemplateService
• LibraryService
• AltoService
• UserService

Clean Up.

The clean up of these files is handled by the: FileCleaningManager.

This service defines the following cron jobs:

• cleanDeliveryFiles
• cleanTemporaryFiles
• cleanExportCaches
• gestDeliveredFiles

Before suppresion of the files not handled by the cron jobs this service provides the option to export a tarball of the files over ftp at the closure of the lot.

La base de données de NumaHOP

NumaHOP utilise MariaDB par défaut. Une autre base de données pourrait théoriquement être utilisée en changeant le driver lors de la compilation.Cependant ce n’est pas recommandé et n’a pas été testé.

La documentation pour MariaDB est disponible ici.

Migration de base de données

NumaHOP utilise liquibase community pour gerer ses migrations. Cela sert aussia l’initialization de la base lors de l’installation.

La documentation de liquibase est disponible ici.

Comment liquibase fonctionne

Liquibase crée deux tables suplémentaires en base DATABASECHANGELOG et DATABASECHANGELOGLOCK. Il utilise celles-ci pour comparer les étapes du journal de changements appliquées avec celles dont il a connaissance et applique les étapes manquantes.

Comment liquibase est mis en place

Le changelog est separee en plusieurs fichiers xml references dans un journal de changements maître. Ces fichiers sont dans le dossier src/main/resources/config/liquibase.

Ajouter une étape

Chaque developpement modifiant la schéma de la base doit ajouter une nouvelle étape et la référencer dans le fichier master.xml.

Les étapes doivent avoir un nom de fichier reflétant les changements du schéma.

Schéma de la base de données

Contribuer a NumaHOP

Si vous pensez a contribuer vous êtes in invité a lire les règles de codage.

L’environnement de développement.

La façon la plus simple de tester vos modifications localement est d’utiliser la version Docker de NumaHOP.

The makefile makes management of the developpement loop rather easy. You can see the full list of rules provided by the Makefile by running make list. If you chose to use the docker you should run make setup-docker before running make build-docker.

You can see the NumaHOP logs with make app-logs.

Also the formating of both the front-end and backend is handled in the makefile with make fmt.

The lifecycle of the composition is manage with app-<action> and env-<action> rules. The actions provided by the makefile are:

• up: Start all containers and start their services.
• stop: Stop the services keeping the containers up.
• down: Stop the containers.
• clean: Same as down detaching the volumes allowing for their deletion.

The makefile also provides ways to manage the mounted volumes in the docker images. For example:

• to reset the db you can run make reset-db.

• to get a clean reinstall of Numahop make reset-all. Note the volumes can only be reset by unmouting them from the containers.

Glossaire

All definition for terms and acronyms used in this documentation.

PGCN: Plateforme de Gestion de Contenu Numérisé. Original name of NumaHOP.

OCR : Optical Character Recognition. Process by which text is recognized on images.

Metadata standards and file formats

METS : Metata-Data Encoding and Transportation Standard An XML standard.

EAD : Encoded Archival Description.

DC : DublinCore Small Set of core metadata for bibliographic units User Guide.

SIP : Submission Information Package A file describing a package to be archived.

AIP : Archival Information Package A file representing an archived packaged.

XSD : XML Schema Definition An XML file describing an XML format allowing an XML parser to verify a file is well formed according to a standard.

MARC: Format for bibliographic data Spec.

ALTO : Analyzed Layout and Text Object.

Protocols

OAI-PMH : Open Archives Initiative Protocol for Metadata Harvesting Spec.

Z39.50 : A protocol to searching and retrieving data in databases between servers over TCP/IP.

IIIF : A REST API for requesting documents and their images SPEC.

NumaHOP Vocabulary

Digitalization service provider: The company(for external digitalization) or service (internal digitalization) doing the digitalization process.

Document Unit: Also referred as Doc Unit or DU; A Document or part of a document for collections to be processed by NumaHOP.

Workflow: Steps the Document Unit takes trough NumaHOP. Can be defined at the project level, or the bundle level.

Notice: Bibliographic meta-datas attached to a DU.

Condition Report: A check of the state of the document after it was manipulated during the digitalization process.

Delivery: The step where the digitalized documents are deposited into NumaHOP and attached to the correct Document Unit.

Import: The step where we create the Document Unit.

Grouping of Document Units

Project: A project is usually a set of documents to be digitalized by the same provider.

Bundle: A physical set of Documentary Units averaging around 20 to 50 documents.

Train: As in digitalization train. Smaller set of document for insurance reason.

All sets of grouping are not necessarily useful for all use cases. For smaller volume of documents in a project the train is not as useful.

Design Pattenrn jargon

DTO: Data Transfer Object an object traveling between the view and the controller.

Service: Class containing business logic.

Repository: Class abstracting a storage method (database, elasticsearch or file system).

Mapper: Class performing mapping from a domain object to another.

Contributeurs

Tech’Advantage: Developpement of Numahop from the initial version.

Persée: Dockerisation of Numahop.

Design Chart

Color palette

Manuels Utilisateurs

Old documentation.

Here you can find the old documentation of NumaHOP writen by Tech’Advange also previously known as Progilone. These pdf manuals are more intended for a user than a developper but are still usefull.

These manuals are written in french.

• MAN002 Administrator Manual
• MAN003 User Manual
• MAN004 Provider Manual
• MAN005 Complete Processus

This documentation is in major party wrong because the swagger generates an incorrect api.json due to the api non-compliance to the oas v3.1 spec.

Modules

Account

/api/rest/reset

POST

Parameters: NONE

Body:

Responses:

/api/rest/account/change_password

POST

Parameters: NONE

Body:

Responses:

/api/rest/authenticate

GET

Parameters: NONE

Body: NONE

Responses:

/api/rest/account

GET

Parameters: NONE

Body: NONE

Responses:

Admin logs

/api/rest/downloadlogsfile

GET

Parameters:

Body: NONE

Responses:

Audit delivery

/api/rest/audit/delivery

GET

Parameters:

Body: NONE

Responses:

Audit doc unit

/api/rest/audit/docunit/{id}

GET

Parameters:

Body: NONE

Responses:

Audit lot

/api/rest/audit/lot

GET

Parameters:

Body: NONE

Responses:

Audit mapping

/api/rest/audit/mapping/{id}

GET

Parameters:

Body: NONE

Responses:

Audit project

/api/rest/audit/project

GET

Parameters:

Body: NONE

Responses:

Audit train

/api/rest/audit/train

GET

Parameters:

Body: NONE

Responses:

Authorization

/api/rest/authorization

GET

Parameters: NONE

Body: NONE

Responses:

/api/rest/authorization/{identifier}

GET

Parameters:

Body: NONE

Responses:

Automatic check

/api/rest/check/auto

GET

Parameters:

Body: NONE

Responses:

Bibliographic record

/api/rest/bibliographicrecord

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/bibliographicrecord/{identifier}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

/api/rest/bibliographicrecord/{id}

GET

Parameters:

Body: NONE

Responses:

Check configuration

/api/rest/checkconfiguration

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/checkconfiguration/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

/api/rest/checkconfiguration/{idDocUnit}

GET

Parameters:

Body: NONE

Responses:

Check

/api/rest/check

GET

Parameters: NONE

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/check/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body:

Responses:

/api/rest/check/pdf/{id}

GET

Parameters:

Body: NONE

Responses:

/api/rest/check/lot_pdf/{id}

GET

Parameters:

Body: NONE

Responses:

/api/rest/check/lot_csv/{id}

GET

Parameters:

Body: NONE

Responses:

/api/rest/check/csv/{id}

GET

Parameters:

Body: NONE

Responses:

Check slip configuration

/api/rest/checkslip_configuration/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

Cines language code

/api/rest/conf/cineslangcode

GET

Parameters: NONE

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

Condition report attachment

/api/rest/condreport_attachment

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body: NONE

Responses:

/api/rest/condreport_attachment/{id}

GET

Parameters:

Body: NONE

Responses:

/api/rest/condreport_attachment/{identifier}

DELETE

Parameters:

Body: NONE

Responses:

Condition report

/api/rest/condreport

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body:

Responses:

/api/rest/condreport/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body: NONE

Responses:

/api/rest/condreport/{identifier}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

/api/rest/condreport/pdf

GET

Parameters:

Body: NONE

Responses:

/api/rest/condreport/csv

GET

Parameters:

Body: NONE

Responses:

Condition report detail

/api/rest/condreport_detail

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body: NONE

Responses:

/api/rest/condreport_detail/{identifier}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

Condition report slip configuration

/api/rest/condreportslip_configuration/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

Csv mapping

/api/rest/csvmapping

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/csvmapping/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

Delivery configuration

/api/rest/delivery_configuration/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

Delivery

/api/rest/delivery

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/delivery/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

/api/rest/delivery/pdf/{id}

GET

Parameters:

Body: NONE

Responses:

/api/rest/delivery/lot_pdf/{id}

GET

Parameters:

Body: NONE

Responses:

/api/rest/delivery/lot_csv/{id}

GET

Parameters:

Body: NONE

Responses:

/api/rest/delivery/csv/{id}

GET

Parameters:

Body: NONE

Responses:

Description property

/api/rest/condreport_desc_prop

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/condreport_desc_prop/{identifier}

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

Description value

/api/rest/condreport_desc_value

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/condreport_desc_value/{identifier}

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

Digital document

/api/rest/digitaldocument/{identifier}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body:

Responses:

/api/rest/digitaldocument

GET

Parameters:

Body: NONE

Responses:

Digital library configuration

/api/rest/conf_digital_library

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/conf_digital_library/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

Digital library diffusion

/api/rest/export/digitalLibrary

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body: NONE

Responses:

Doc property type

/api/rest/docpropertytype

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/docpropertytype/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

Doc unit

/api/rest/docunit

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body:

Responses:

/api/rest/docunit/{identifier}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

/api/rest/docunit/initHistory

GET

Parameters:

Body: NONE

Responses:

Export cines

/api/rest/export/cines

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body:

Responses:

/api/rest/export/cines/{id}

POST

Parameters:

Body:

Responses:

/api/rest/export/cines/{identifier}

GET

Parameters:

Body: NONE

Responses:

/api/rest/export/cines/regenerateMets

GET

Parameters:

Body: NONE

Responses:

Export

/api/rest/export/rdfxml

GET

Parameters:

Body: NONE

Responses:

/api/rest/export/ead

GET

Parameters:

Body: NONE

Responses:

/api/rest/export/csv

GET

Parameters:

Body: NONE

Responses:

Export ftp configuration

/api/rest/exportftpconfiguration

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/exportftpconfiguration/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

Export internet archive

/api/rest/internet_archive/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body:

Responses:

/api/rest/internet_archive

GET

Parameters:

Body: NONE

Responses:

Export omeka

/api/rest/export/omeka

GET

Parameters:

Body: NONE

Responses:

POST

Parameters:

Body: NONE

Responses:

File cleaning

/api/rest/filecleaning/deleteorphans

GET

Parameters:

Body: NONE

Responses:

Files gestion config

/api/rest/filesgestionconfig

POST

Parameters: NONE

Body:

Responses:

/api/rest/filesgestionconfig/{id}

POST

Parameters:

Body:

Responses:

/api/rest/filesgestionconfig/{idLibrary}

GET

Parameters:

Body: NONE

Responses:

Ftp configuration

/api/rest/ftpconfiguration

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/ftpconfiguration/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

Health

/api/rest/health

GET

Parameters: NONE

Body: NONE

Responses:

Help page

/api/rest/help

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/help/{id}

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

DELETE

Parameters:

Body: NONE

Responses:

Image metadata

/api/rest/imagemetadata

GET

Parameters:

Body: NONE

Responses:

POST

Parameters: NONE

Body:

Responses:

/api/rest/imagemetadata/{id}

POST

Parameters: NONE

Body:

Responses:

Import

/api/rest/import

POST

Parameters:

Body:

Responses:

Import report

/api/rest/importreport

GET

Parameters:

Body: NONE

Responses: