Application Development

Getting Started

Applications within SAGE3 are similar to an Application on an Operating System. Within SAGE3 Applications exist on Boards and can be manipulated by various clients. There are two types of Applications: Plugin Applications and Integrated Applications.

• Plugin Applications: Developed outside of the SAGE3 source code and are uploaded to a SAGE3 server.
• Integrated Applications: Developed within the SAGE3 source code.

Plugin Applications allow developers to develop their application however they want. It can be a Vanilla HTML/JS webpage, React App, Angular, Svelt, Vue...any framework is supported. Developers can also use any library they desire without the worry of cross pollution without Applications. With the @sage3/sageplugin (optional) you can also sync your applications state across clients.

Integrated Applications are developed within the SAGE3 source code and are built/integrated into the bundled SAGE3 Webapp. Integrated applications have more access to SAGE3 features but are also more restricted.

Currently the best way to develop a new app for SAGE3 is to use the @sage3/sageplugin npm package. This way you don't have to work with the SAGE3 source code and can develop your app in which ever environment of your choosing, we recommend Vite. You can also ignore using the @sage3/sageplugin package completely and just develop a vanilla webapp or webpage.

Plugin Application

IMPORTANT

The one key thing to ensure are imports in your distributed index.html (assets, scripts, styles, etc.) are referenced with relative paths.

• Instead of <script src="script.js">
• It has to be relative, <script src="./script.js"/>

The Vite example here configures this in step 3, with the vite.config.js file, so you don't have to worry about it. If you use your own framework you will have to ensure the paths are relative.

Webapp in Typescript

You do not have to use the @sage3/sageplugin library, but without it you will not be able to sync state across clients. Using the @sage3/sageplugin library enables developers to leverage SAGE3 to sync state for their Plugin App across clients.

To get started you can make your application using which ever framework you desire but we recommend ViteJs

Ensure you have NodeJs installed and create a new project with Vite.

1. Create a new project for you plugin app. I'm using Vanilla (No framework) and Typescript, but you don't have to.

npm create vite@latest my-first-sage-app
> Vanilla
> Typescript

2. Initial Setup

cd my-first-sage-app
npm install

3. Configure Vite by updating or creating a file in the root folder of the project named vite.config.js, or vite.config.ts for Typescript, and copy the text below into it. If the file already exists just update the config object with base and outDir property. This step ensures that when you go to build your plugin app the index.html file will have relative paths to the various references.

import { defineConfig } from "vite";

export default defineConfig({
  base: "./",
  build: {
    outDir: `./dist/sage3_plugin_app`,
  },
});

4. Create you app within the src folder. Vite comes with some boiler plate code to get you started.
5. (OPTIONAL) Install the @sage3/sageplugin library. It provides an API to allow the apps developed this way to communicate with the SAGE3 ecosystem.

npm install @sage3/sageplugin

6. (OPTIONAL) Utilize the @sage3/sageplugin library to communicate with the SAGE3 ecosystem. Here is a basic example to share the state of a counter across mutiple clients:

// Import the Package
import { SAGE3Plugin } from "@sage3/sageplugin";

// You Plugin app state type
type CounterState = {
  count: number;
}

// The local variable we a syncing for this example.
const counter = 0;

// Intialize the SAGE3Plugin.
// Only intalize once. Utilize it as a singleton throughout your app.
const s3api = new SAGE3Plugin<CounterState>();

// Subscribe to updates from the SAGE3 server when other clients update the state.
s3api.subscribeToUpdates((state) => {
  if (state.data.state.count) {
    counter = state.data.state.count;
  }
});

// Push an update to the SAGE3 server.
s3api.update({ state: { count: counter + 1 } });

// Can also reference the current state object from the S3Plugin
const currentCount = s3api.state.data.state.count;
s3api.update({ state: { count: currentCount + 1 } });

7. Build you app for deployment

npm run build

8. If successful, there should be a folder named sage3_plugin_app in the dist folder. It should have an index.html file.

/dist
  /sage3_plugin_app
    /index.html
    /...other files/folders related to your plugin app

9. Zip the sage3_plugin_app folder using your Operating System's tools.
10. Skip to the Upload Section.

Webapp in plain Javascript

If you would rather use plain Javascript instead of Typescript, you can follow the steps below. You do not have to use the @sage3/sageplugin library, but without it you will not be able to sync state across clients. The @sage3/sageplugin library enables developers to leverage SAGE3 to sync state for their Plugin App across clients.

Find a complete example here: https://github.com/SAGE-3/plugins/tree/main/counter-js

1. Create a new folder for you plugin app and copy your HTML, CSS, and JS files into it. For instance:

• index.html
• index.js
• style.css

Make sure your HTML file has relative paths to your JS and CSS files, so they can be referenced correctly when the app is uploaded to SAGE3. Load your JS as a module, so you can use ES6 imports.

    <script type="module" src="index.js"></script>

2. (OPTIONAL) Include the @sage3/sageplugin library in your index.js file. It provides an API to allow the apps developed this way to communicate with the SAGE3 ecosystem.

import { SAGE3Plugin } from "https://unpkg.com/@sage3/sageplugin@0.0.15/src/lib/sageplugin.js";

This load the sageplugin from a CDN server, so you do not have to install it locally.

3. (OPTIONAL) Utilize the @sage3/sageplugin library to communicate with the SAGE3 ecosystem. Here is a basic example to share the state of a counter across mutiple clients:

// The local variable we are syncing
let counter = 0;

// Intialize the SAGE3Plugin.
// Only intalize once. Utilize it as a singleton throughout your app.
const s3api = new SAGE3Plugin();

// Subscribe to updates from the SAGE3 server when other clients update the state.
s3api.subscribeToUpdates((state) => {
  if (state.data.state.count) {
    // update the local counter value
    counter = state.data.state.count;
    // update the DOM
    counterValue.innerHTML = counter;
  }
});

// To increment the value of counter
incrementBtn.addEventListener("click", () => {
  counter++;
  counterValue.innerHTML = counter;
  // Push an update to the SAGE3 server.
  s3api.update({ state: { count: counter } });
});

4. Zip your application folder using your Operating System's tools.
5. Skip to the Upload Section.

Unity

You can also upload a Unity WebGL application through the SAGE3 Plugin Interface. We currently don't have a way to sync state across clients but it is a way to share your Unity Applications through SAGE3.

1. Set you Build Target to WebGL.

2. Disable the Compression Format. Under Player Settings > Publish Settings > Compression Format > Disabled

3. Build your Unity Project.

4. Zip you Unity Project's Build folder from step 3.

5. Skip to the Upload Section.

Upload to a SAGE3 Server

1. Open SAGE3 and go to the server of your choice.
2. Open the Plugin Menu from the lower left corner Main menu.

3. Under Upload select the zip file created above. Enter you Plugin's name, and your Plugin's Description. Click Upload.

4. After your plugin uploads it should appear in the Your Plugins list.

5. Navigate to a board of your choosing.

6. Within the Plugin Panel you should now see your newly uploaded Plugin.

7. Click your Plugin's button from the Plugin Panel to open it on the board.

Integrated Application

An Integrated Application is a SAGE3 App that is contained within the SAGE3 Source code. It is fully integrated within SAGE3 and has access to a lot more features than a Plugin Application. Documentation explaining how to develop an Integrated Application is ongoing.

Setup

• Clone the repository: https://github.com/SAGE-3/next
• Have Nodejs, Docker with Docker Compose, and python3 installed.
• We use Yarn for package management. You can install it with npm install -g yarn

Backend

The backend runtime is composed of Redis and a Jupyter instance. The Jupyter instance is used to run the Python code that is uploaded to the SAGE3 server. The Redis instance is used to store the state of the applications and the SAGE3 collections.

1. Navigate to the backend folder.

2. Run docker-compose up -d -f docker-compose-backend.yaml to start the backend runtime.

   • or use the Backend script
   • you can check that redis and jupyter are running with docker ps or open http://localhost in your browser. The jupyter token is stored in redis under the key config:jupyter:token, you can get it with redis-cli get config:jupyter:token. You can also check the file info.json in the jupyter container: docker exec -it deployment-jupyter-1 cat /conf/info.json

Web development

To develop the web application you will need to run the frontend and backend in development mode. The frontend will be served by React and the backend will be served by NodeJS. Both are written in Typescript and compiled to Javascript.

1. Open a Terminal and navigate to the webstack folder.
2. Run yarn install to install the dependencies.
3. The first time setting your development environment, run yarn stage to deploy some files to the dev server.
4. Run yarn start to start the development server. The server is an Express-based web server running on port 3333 by default.
5. Open a second Terminal and navigate to the webstack folder.
6. Run yarn webapp to run the React development server, which runs on port 4200 by default. This server will proxy requests to the backend server.
7. Open a browser and navigate to http://localhost:4200. You should see the SAGE3 login page.

Note: depending on your operating system, you can open Terminal applications as described above, or use VsCode's built-in Terminal.

SAGE3 provides a utility to scaffold a new Integrated Application. It will create a new folder in the webstack/libs/applications/src/lib/apps folder with the necessary files to get started.

1. Open a Terminal and navigate to the webstack folder.
2. run yarn new-app and follow the prompts.

$ yarn newapp
✔ What name would you like to use for the SAGE3 application? · NewApp
✔ What is the name of the developer? · SAGE3 Team
✔ Provide the name of the state variable · value
✔ Which data type would your application use? (others can be added later) · number
✔ Provide the default value of the state variable? · 13
Schema> {
  name: 'NewApp',
  username: 'SAGE3 Team',
  statename: 'value',
  statetype: 'number',
  val: 13
}
CREATE libs/applications/src/lib/apps/NewApp/NewApp.tsx
CREATE libs/applications/src/lib/apps/NewApp/index.ts
CREATE libs/applications/src/lib/apps/NewApp/styling.css

3. You might to restart yarn webapp to load the new genrated files.
4. Open your favorite code editor and open webstack/libs/applications/src/lib/apps/NewApp folder.

• index.ts is a Typescript file that describe the schema (exported as type state) of the app (using the Zod library). It also defines the initial state of an instance of the application and the name of the application. The generator automatically adds the new type in SAGE3 codebase (files in webstack/libs/applications/src/lib/apps/).
• styling.css is a CSS file that contains the styling of the application. It is automatically imported in the NewApp.tsx file.
• NewApp.tsx is a React file that defines the AppComponent of the application and the ToolbarComponent of the application. It automatically imports the index.ts file with the schema of the application.
• While in development mode, your application will be listed in the SAGE3 application panel. You can click on it to open a new instance of the application.
• If your application requires new packages, just add them with yarn add <package> and restart the backend development server and the React development server. Make sure to add @types/<package> if the package has typescript definitions, as a 'development' dependency.
• Check the Counter application as a example on how to manage state and communicate with the backend (webstack/libs/applications/src/lib/apps/Counter/Counter.tsx)

Python development

The python development happens mostly in and around the proxy that communicate between the NodeJS webserver, Redis and the Jupyter instance.

1. Open a Terminal and navigate to the foresight folder.
2. The easiest system is to setup a conda environment dedicated to SAGE3 development:

• with conda create -n s3-next python=3.8
• install the dependencies with pip install -r requirements.txt
• load the environment variables into you Bash shell with source _env.sh

3. Start the proxy with python proxy.py

• the proxy outputs its logs in the proxy.log file (keep an eye on it if you have issues with tail -f proxy.log)

4. To define custom behavior and functions, define or extend smatbits on your application.
5. To send and receive message between the React frontend and the Python backend, add the executeinfo field to your application schema.

  executeInfo: z.object({
    executeFunc: z.string(),
    params: z.any(),
  }),