Top
# HALO 3.3 / HALO Link 2.3 / "Fiesta"

### General

* The .NET runtime for the backend servers has been updated to .NET
  Core 3.1. The HALO and processing node applications already made
  this update in HALO v3.2. ASP.NET Core has also been updated.

### Identity provider and login

* The IdentityServer library has been updated to the latest major
  revision.  A notable change in this update is that URLs in the
  OpenID discovery document are now generated using the
  X-Forwarded-Proto and X-Forwarded-Host HTTP headers. If you use HALO
  Link, updating to HALO Link 2.3 is required for this to work
  correctly alongside HALO 3.3, otherwise the OpenID discovery
  document will contain in correct `http://` URLs.

### GraphQL server

* The GraphQL API's schema has been updated to be more consistent
  in type names and allow clients to more easily consume our GraphQL
  schema. For more details see
  C:/ProgramData/Indica Labs/Documentation/IndicaLabs.ApplicationLayer.Halo.GraphQLApi/UPGRADE_NOTES/HALO-3.3-GraphQLApi.md

* Permissions may now be managed for many users at once with user groups.
  Users may be added to a group and a study may be shared with a group with
  a given permission level, just like sharing a study with a user.

# HALO 3.2 / HALO Link 2.2 / "Elote"

## Architectural notes

### General

* All core services with a web interface now serve `/version` and
  `/health` endpoints that can be used to confirm that the server is
  live.  `/health` may be used at some future point to perform
  additional checks such as database connectivity, availability of
  other dependencies, etc.

* Usability of `config_docs` has been improved. In particular if your
  configuration cannot be parsed, the summary of available
  configuration options will be written to C:/ProgramData/Indica
  Labs/Logs/<service/config_docs.yml. Previously it could be difficult
  to determine what configuration options are valid if your current
  configuration files are invalid, because it was not possible to run
  the `config_docs` CLI in this case.

### Identity provider and login

* HALO Link, HALO, the GraphQL Workbench, the File Monitor, the
  Processing Node, and the Training Agent and Compute Coordinator are
  all now OIDC clients.

  - In the case of HALO Link this is the same as
    in HL 2.1 - it is an authorization code flow client.

  - For the GraphQL workbench, this is as in HL 2.1, with the added
    change that the GraphQL workbench can now be straightforwardly
    accessed behind the HALO Link App Server reverse proxy, i.e.
    by accessing `https://<HALO Link hostname>/graphql` in the browser.
    GraphQL redirect URIs for the identity provider should be changed
    from e.g. `http://localhost:4030/login-callback` to
    `https://<HALO Link hostname>/graphql/login-callback`.

  - The File Monitor and Processing Node are now machine-to-machine
    clients using the client credentials flow. This means that they
    authenticate with the identity provider using a shared secret
    (stored in the IdP configuration and in the File Monitor or
    Processing Node configuration) which can be exchanged for an
    access token at any time -- these access tokens allow these
    services to make API calls to the GraphQL or gRPC APIs without an
    associated user account. When installing everything on the same
    machine, this is configured automatically. When installing
    procesisng nodes on separate machines, the `reconfigure --script
    AddResearchServiceClient` script on the IdP may be used to
    provision a client ID and secret. The resulting client
    configuration looks like, for instance:

    ```yaml
    # C:/ProgramData/Indica Labs/Configuration/IndicaLabs.ApplicationLayer.Halo.IdentityProvider

    identity_provider:
      clients:
      - !OidcClient/ClientCredentials
        id: analysis_LAPTOP-KMQI9SLO
        scopes:
        - serviceuser
        - graphql
        - grpc
        require_client_secret: true
        secrets:
        - secret: JnDrpPa244JNmUsWIfiDXw==
        implementation: !OidcImpl/HALO {}
    ```

    To add a new processing node (for example), copy this section and
    use a new client ID and shared secret. You can generate an
    appropriate secret using the `generate_secret` CLI on the identity provider:

    ```
    "C:/Program Files/Indica Labs/HALO Identity Provider/IndicaLabs.ApplicationLayer.Halo.IdentityProvider.exe" generate_secret
    ```

  - The Processing Node additionally passes connection information to
    its job workers when spawning them, including an access token. A
    Halo.exe.config file should no longer be needed by the job workers
    -- their connection configuration is instead derived from the
    processing node configuration (C:/ProgramData/Indica Labs/Configuration/Halo.AnalysisAgent.Server).

  - Similar changes have been made for the HALO AI training agent's
    API connections to use OIDC access tokens instead.

  - HALO may act either as an authorization code flow client or a
    client credentials client. The authorization code flow is expected
    for a multi-user deployment and achieves a shared single-sign-on
    experience with HALO Link by using the default web browser as a
    user agent. For "HALO Classic" single workstation deployments,
    prompting for credentials doesn't make sense so a client
    credentials flow is configured instead. This is configured
    automatically depending on whether you select "full install"
    (client credentials flow) or "HALO client only" (authorization
    code flow) during HALO installation.

  - The `add_user` and `reset_password` CLIs have been removed from
    other services and added to the Identity Provider instead. Note
    that you must now specify a `--client` (e.g. `--client=halo_link`)
    to inform the IdP about what database connection to use.

* Identity provider configuration has changed so that a different
  database may be used depending on which client is calling. This
  change allows future deployments to perform sign-ins for HALO Link
  or HALO AP (different database schemas) using the same IdP
  instance. This is configured via a new required `implementation`
  section for each client, e.g.

  ```yaml
  - !OidcClient/AuthorizationCodeFlowClient
    id: halo
    implementation: !OidcImpl/HALO
      persistence:
        db:
          username: indica
          ...
  ```

  In a typical HALO + HALO Link deployment all `implementation`
  sections will be the same.  In this case it will fall back to a
  top-level `persistence:` section if present -- this will typically
  be derived from Shared Configuration. Therefore typically the only
  additional configuration needed here is

  ```yaml
  - !OidcClient/AuthorizationCodeFlowClient
    ...
    implementation: !OidcImpl/HALO {}
  ```

* Configuration for login providers has changed to be more
  uniform. Rather than a separate `login: methods:` section,
  all providers including local logins and LDAP / Active Directory logins
  are configured under the `identity_provider: login_providers:` section:

  ```yaml
  # C:/ProgramData/Indica Labs/Configuration/IndicaLabs.ApplicationLayer.Halo.IdentityProvider/local-production.yml
  identity_provider:
    login_providers:
    - !IdP/Credentials
      scheme: local
      client_id: local
      display_name: Local
      settings: !Login/Local {}

    - !IdP/Credentials
      scheme: activedirectory
      client_id: activedirectory
      display_name: Active Directory
      settings: !Login/ActiveDirectory
        domain: indicatest.com
        email_domain: indicalab-ldap1.com
        uri: ldap://10.1.10.58
        port: 389

    - !ExternalIdP/OpenIdConnect
      client_id: server.code
      client_secret: secret
      authority: https://demo.identityserver.io
      display_name: Identity Server demo
      response_type: code
      scheme: IdentityServer4
  ```

  Since HALO now uses the IdP to manage logins, you no longer need
  configuration in the gRPC API to manage active directory logins
  -- it will not be used.

### "Gateway" and TLS termination

To contact the identity provider securely, an HTTPS connection is
required, which means that TLS certificates must be set up somewhere
in a given deployment, and there must be an HTTPS web server directing
traffic to the IdP. When HALO Link is installed, the HALO Link app
server can take this role. This is unchanged from HALO 3.1 / HL 2.1 --
when HALO Link is installed the HALO Link app server is a natural
place to terminate TLS and reverse-proxy to other servers (though this
is by no means the only possible topology).

However, now that more components such as HALO, the processing node,
and the file monitor rely on the identity provider, a HALO
installation without HALO Link also requires this
functionality. Therefore HALO ships with a new component, the "Indica
Labs API Gateway" service (IndicaLabs.ApplicationLayer.Gateway,
C:/Program Files/Indica Labs/API Gateway) which can be configured with
HTTPS certificates and act as a reverse proxy.

The HALO Link app server and the API Gateway cannot typically both be
running on the same machine, since they are both attempting to bind
port 443 for HTTPS web traffic. Therefore when HALO Link is installed,
or if HL is already installed on a machine when updating to 3.2, this
service will be configured in a "Disabled" state so that it does not
normally launch in the background. In deployments where HALO Link is
not involved, however, it is necessary for the API gateway service to
be running and configured with suitable certificates in order for
other system components such as HALO clients or processing nodes to be
able to contact the IdP.

For simple installations, this should all be handled automatically,
but it is important to understand the relationship between these
components when troubleshooting OIDC connectivity issues. In the
future, this gateway service may take on additional responsibilities
for communication between system components.

### File monitor / image access

* The file monitor now saves image data to the database using a
  graphql connection rather than direct database access. It
  authenticates using an OIDC client credentials flow, which should
  be configured automatically by a "full install".

* For performance reasons the cached image preview blobs (thumbnails
  and label images) have been moved from the `image` table into their
  own `imagepreview` table. Existing data is not deleted or moved by
  the migration to avoid blocking the update. Additionally the blobs
  are now stored as JPEGs instead of bitmaps to reduce storage space
  and network bandwidth.

* Because of the change in storage, preview blobs will need to be
  re-generated or imported as an offline step after migrating the
  database. The file monitor configures this by default:

  ```yaml
  # C:/ProgramData/Indica Labs/Configuration/IndicaLabs.ApplicationLayer.FileMonitor/default.yml

  monitor:
    import_image_blobs:
    - !ImageBlobs/Legacy {}
  ```

  In addition to the file monitor's normal operation, this
  configuration instructs it to move existing thumbnails from the
  `image` table to the `imagepreview` table, converting the bitmaps to
  JPEGs in the process. Note that this requires direct database access
  to query the existing blobs from the `image` table (and remove them
  when the import is done).

* There is a similar option that can be used for generating missing
  thumbnail / label previews by attempting to open the actual image
  files. This can be useful in the case of merged databases or imports
  from third party image management systems.

  ```yaml
  # C:/ProgramData/Indica Labs/Configuration/IndicaLabs.ApplicationLayer.FileMonitor/local-production.yml
  monitor:
    import_image_blobs:
    - !ImageBlobs/Missing {}
  ```

  Note that this is a much slower process than `!ImageBlobs/Legacy`
  since it needs to open and read each image with missing previews.

* To disable automatic preview import / generation in the file
  monitor, you can set:

  ```yaml
  # C:/ProgramData/Indica Labs/Configuration/IndicaLabs.ApplicationLayer.FileMonitor/local-production.yml
  monitor:
    import_image_blobs: []
  ```


### Processing node

* See the identity provider notes above. In particular note that the
  processing node -> graphql connection no longer uses the
  "`service_users`" approach where each processing node needs to be
  configured with the GraphQL server under `login:
  service_users:`. Instead processing node authentication is
  configured in the identity provider as a client credentials client.

* Database access done by the analysis job workers is now also done
  over GraphQL instead of gRPC.

# HALO 3.1 / HALO Link 2.1 / "Dune"

* The Identity Provider service was added for managing federated
  sign-on.

* The Processing Node was changed to use GraphQL instead of gRPC for
  database access, and changed to use the server framework used by
  GraphQL / gRPC / etc.

### HALO Link legacy API

The HALO Link legacy API, used for mediating database access since the
earliest release of HALO Link, has been removed. All its
responsibilities are instead handled through the GraphQL API. This
makes for less deployment complexity, a smaller HALO Link size on
disk, and means that anything HALO Link can do should at this point be
achievable through the GraphQL API.

## Database updates

With each release it is necessary to make some changes to how the
Indica Labs database is structured to add new functionality and
address performance issues. Here is a summary of the database
migrations performed during the 3.1 -> 3.2 update process.

| Migration ID | Purpose |
| --- | --- |
| 20200814155250_Schema_AddImagePreviewTable | Adds a separate table for storing "thumbnail" and "label" preview images. See above under "File monitor / image access" |
| 20200817172442_Data_RemoveStudyClosureDuplicates | These two migrations are used to ensure and enforce a consistent structure for the "closure table" for studies, a denormalization used to help efficiently check whether one study is a descendant or ancestor of another. |
| 20200818200309_Schema_AddStudyClosureUniqueConstraint | Same as above. |
| 20200826152658_Schema_AddUserGroupsAndPrivilegeSets | Adds new models for managing groups of users, a preview feature in HALO Link 3.2. |
| 20200902210458_Schema_MakeClassifierXMLNonNull | Helps to ensure that only well-formed classifiers are stored in the database. |
| 20200910035657_Schema_AddAnalysisJobSetTable | Adds new models for associating many analysis jobs as part of the same group. Used for new HALO AI validation features. |
| 20200911223933_Schema_RemoveStudyTriggers | Removes some database triggers and stored procedures used for maintaining the structure of the study hierarchy which are no longer necessary. |
| 20200914192936_Schema_SetMaxAnnotationSetNameSize | Enlarges the annotationset.Name column from 95 to 255 characters. |
| 20200924220212_Schema_MakeRegistrationForeignKeyOnJobControlSetNull | Prevents the history of a registration attempt from being deleted from the database when a registration is deleted. |
| 20200929203936_Schema_DropStudyPermissionViews | Removes some database views for permission checks that are no longer used. |