proPilot installation and configuration

This document explains how to install proPilot on a machine.
Please read the instructions carefully, and follow the steps in the defined order.

Containerized deployment

The recommended way to deploy proPilot is to use the containerized approach.

But as explained in the requirements document, proPilot supports multiple options to handle SSO, so the identity server is not part of the installation file hereafter. To install Keycloak or any other Identity server solution, refer to the installation guide of your vendor.

1. Create a directory on the target host (/opt/dfakto/proPilot, C:\Program Files\dFakto\proPilot, …)

2. Create a file named docker-compose.yml with the following content as a base.

services:

  propilot-db:
    image: docker.io/library/mariadb:11.8.2
    container_name: propilot-db
    restart: always
    environment:
      MARIADB_ROOT_PASSWORD: 'rootPassHere'
      MARIADB_DATABASE:      'propilot_global'
      MARIADB_USER:          'propilot'
      MARIADB_PASSWORD:      'propilotUserDBPassHere'
    volumes:
      - mariadb_data:/var/lib/mysql
      - ./docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d:ro
    command:
      - --sql-mode=ANSI_QUOTES
      - --lower_case_table_names=1
    healthcheck:
      test:
        [
          "CMD",
          "healthcheck.sh",
          "--connect",
          "--innodb_initialized"
        ]
      interval: 10s
      timeout: 5s
      retries: 5

  propilot-api:
    image: quay.io/dfakto_org/propilot:${PROPILOT_API_VERSION}
    container_name: propilot-api
    environment:
      AppSettings__connectionString: "Server=propilot-db;Port=3306;Database=propilot_global;User Id=propilot;Password=propilotUserDBPassHere"
      AppSettings__allowedOrigin: "http://front-end-separate-domain-if-necessary:4200"
      
      DFAKTO_PROPILOT_HTTP_PORTS: 8080
      Logging__LogLevel__Default: Information
      
      # First WriteTo: Console
      Serilog__WriteTo__0__Name: Console
      Serilog__WriteTo__0__Args__outputTemplate: "[{Timestamp:yyyy-MM-dd HH:mm:ss.fff} {Level}] {Message:lj} <s:{SourceContext}>{NewLine}{Exception}"
      Serilog__WriteTo__0__Args__theme: "Serilog.Sinks.SystemConsole.Themes.AnsiConsoleTheme::Code, Serilog.Sinks.Console"

      # Second WriteTo: File
      Serilog__WriteTo__1__Name: File
      Serilog__WriteTo__1__Args__path: "/var/log/propilot/propilot.txt"
      Serilog__WriteTo__1__Args__rollingOnFileSizeLimit: "true"
      Serilog__WriteTo__1__Args__fileSizeLimitBytes: "10000000"
      Serilog__WriteTo__1__Args__retainedFileCountLimit: "10"
      Serilog__WriteTo__1__Args__formatter: "Serilog.Formatting.Json.JsonFormatter, Serilog"
  
      identityServer__Host: 
      identityServer__clientId: 
      identityServer__ValidAudience: 
      # -- set the 2 variables only if you use the master mode
      #identityServer__keycloakHost: https://accounts.your-keycloak.com
      #identityServer__keycloakSecret: the-secret-configured-in-keycloak
      identityServer__FirstAdminEmail: your-proPilot-admin-email@your-company.com
      identityServer__UserIdInIdentityServerClaim: "sub"
      identityServer__UserNameClaim: "preferred_username"
      identityServer__DisplayNameClaims: "given_name, family_name"
      
      Sentry__Dsn: ""
      #Sentry__Dsn: https://yourPublicKey@o0.ingest.sentry.io/yourProjectId
      #Sentry__Environment: development
      
      Smtp__SmtpDeliveryMethod: SpecifiedPickupDirectory
      Smtp__From: "./"
      Smtp__Host: null
      Smtp__Port: null
      Smtp__UserName: null
      Smtp__Password: null
      AppFolders__TreeNodeCodeFileBasePath: "/data/propilot/nodes"
      AppFolders__TempFolder: "/data/propilot/temp"
      AppFolders__PrivateKeysPath: "/data/propilot/keypath"
    depends_on: 
      propilot-db:
        condition: service_healthy
    restart: always
    volumes:
      - propilot_data:/data/propilot
      - propilot_logs:/var/log/propilot

  propilot-ui:
    image: quay.io/dfakto_org/propilot-ui:${PROPILOT_UI_VERSION}
    container_name: propilot-ui
    environment:
      NG_APP_USE_EXTERNAL_API: "false"
      NG_APP_PROPILOT_API_URL: http://propilot-api:8080
      NG_APP_APPLICATION_AVAILABLE_LANGUAGES: "en,fr"
      HTTP_PORT: 80
    ports:
      - "4200:80"
    restart: unless-stopped
    
volumes:
  mariadb_data:
  propilot_data:
  propilot_logs:

Adaptations

adapt the docker-compose file to suit your needs.
You should at least search for “your-” in the above example, and replace the default values with valid values.

For instance, you may want to configure Keycloak (or other identity server).

Here are the main elements to adapt

Database

The following keys must be set with highly secured passwords.

• MARIADB_ROOT_PASSWORD

• MARIADB_PASSWORD

Logging

The application writes valuable information about the execution flow. It can be sent to various output channels like the console or log files. ProPilot uses Serilog to write the application traces.

• MinimumLevel: Indicate the level of log we want to store. From low to high, these are Verbose, Debug, Information, Warning, Error and Fatal

• rollOnfileSizeLimit: Indicate if we want to create a new log file when the current one reaches its size limit

• fileSizeLimitByte: Indicate the size limit of a log file. Once this size is reached, a new file will be created if the rollOnfileSizeLimit is set to true

• retainedFileCountLimit: Indicate how much file we should have, we start overriding the first log file.

• option : formatter: The formatter decides the format of the logs (text, json, …)

For more details about the configuration:

https://github.com/serilog/serilog-settings-configuration

Main application settings

• AppSettings__connectionString

• AppSettings__allowedOrigin
  Configure this key if proPilot front-end is deployed on a separate domain (compared to the API).
  

Identity server (OIDC)

Here are the keys to configure if you want to use an OIDC server for your SSO:

• identityServer__Host
  SSO server address

• identityServer__keycloakHost
  (Master mode only)

• identityServer__keycloakSecret
  (Master mode only)

• identityServer__FirstAdminEmail
  use the email of a user that exists in your SSO. It will be the first admin in proPilot (for your first connections, tests, and in-app configurations)

• identityServer__UserIdInIdentityServerClaim
  Claim containing the id of the user in the OIDC. Usually “sub”

• identityServer__UserNameClaim
  Claim containing the unique username that will be displayed in proPilot.
  You can use the same claim as the UserIdInIdentityServerClaim if you don’t have any specific human-readable identification for your users.

• identityServer__DisplayNameClaims
  You can supply a single claim, or a comma-separated list of claims to build the display name (e.g. the first name and family name). If you don’t have any particular display name, you can reuse the UsernameClaim.

SMTP

• SMTP configuration (can be “Network" or “SpecifiedPickupDirectory“)

  • if SpecifiedPickupDirectoryis chosen, you must configure the Smtp__PickupDirectoryLocation (and leave the other field to null: host, port, username, password)

  • if Network is chosen, you must leave the Smtp__PickupDirectoryLocation to null, and configure the other fields (host, port, username, password).

Others

• Replace the “accounts.yourCompany.com” by the correct host.

• Replace ${PROPILOT_API_VERSION} by the correct version (must be supplied by dFakto)

• Replace ${PROPILOT_UI_VERSION} by the correct version (must be supplied by dFakto)

Sentry (optional)

• Sentry (see next section)

Sentry

Sentry is a developer-first error tracking and performance monitoring platform. This configuration is optional. See official documentation for more information.

How to disable Sentry?

To disable sentry, Set the DSN to an empty string, and don’t configure the other sentry keys. Here is a configuration sample to achieve that situation

Sentry__Dsn: ""

How to enable Sentry?

To enable sentry, you must set the required keys

Sentry__Dsn: https://yourPublicKey@o0.ingest.sentry.io/yourProjectId
Sentry__Environment: development

sentryDsn - Sentry Data Source Name. Here how to get your project sentry DSN

environment - String that identifies the environment in sentry (Ex: “development”, “production”, “staging”)

DB initialization

Privilege-by-privilege justification

in the same folder as your docker-compose file you must create a sub folder named “docker-entrypoint-initdb.d”.

Inside that sub-folder, create a file named “init.sql”, with the following content:

-- Create user if it doesn't exist and grant global privileges
CREATE USER IF NOT EXISTS 'propilot'@'%' IDENTIFIED BY 'same_password_as_MARIADB_PASSWORD_in_docker_compose_file';
GRANT ALL PRIVILEGES ON *.* TO 'propilot'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;

Front-End Configuration

There are two methods to configure the connection to the back-end.

When the back-end is hosted at the same URL as the front-end:

Set the environment variable:

NG_APP_USE_EXTERNAL_API to "false".

Additionally, the ProPilot API URL should point to the container address:

NG_APP_PROPILOT_API_URL: http://propilot-api:8080

When the back-end is hosted at a different URL than the front-end:

Set the environment variable

NG_APP_USE_EXTERNAL_API to "true".

The ProPilot API URL should then be set to the appropriate external URL:

NG_APP_PROPILOT_API_URL: https://my-backend-url.com

Is possible to specify the front-end used port via env variable HTTP_PORT. Default value is 8080.

To specify the languages available for display on the front-end, define the following environment variable:

NG_APP_APPLICATION_AVAILABLE_LANGUAGES: "en,fr"

The supported languages include English (en), French (fr), and Dutch (nl)

Launch the application

When launched, the application will automatically create the necessary tables.

⚠️ As a Reminder, you won’t be able to connect to the application if you haven’t configured the admin user email address. This first admin user must exist in Keycloak (or in your SSO solution). This is because we need a first admin user to enter the application.
See docker-compose file above.

identityServer__FirstAdminEmail: your-proPilot-admin-email@your-company.com

Configure operational parameters in the global database

Technical configuration files (appsettings.json) were designed to launch the application (DevOps oriented), but when the application is up, operational information must also be configured.

You can interact with the DB using a command similar to this one. You will be prompted to enter the root password, and will then be able to type SQL commands.

docker exec -it propilot-db mariadb -u root -p

Connect to the correct DB:

MariaDB [(none)]> USE propilot_global;
Database changed
MariaDB [propilot_global]> SHOW TABLES;

Use standard SQL commands to update the “Config” table in the global database, and setup the following entries according to your needs.

-- NOTE: replace <key> & <value> according information in the table below
UPDATE config SET Value = '<value>' WHERE config.Key = '<key>';

Test the back-end

1. Make sure that the application is started

2. The following url should work (replace localhost by your domain name if necessary):
   http://localhost/api/info

3. The page is not html formatted but it must contain the back-end version of proPilot (e.g. ProPilot API VX.X.X)