daniel543x/artcraft
1
0 Fork 0
mirror of https://github.com/storytold/artcraft.git synced 2026-11-07 02:21:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

updated setup README

Browse Source
This commit is contained in:
Brandon Thomas committed 2023-09-19 20:01:38 -04:00
1 parent 14d028ea8a
commit f6ee06a72f
2 files changed
+130 -62
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

_develop/docs/README.md
+3 -2
View File
@@ -1,8 +1,9 @@
README
======
This directory contains a collection of documentation on running the backend.
This directory contains a collection of (hopefully organized and up to date) documentation on
running the various backend pieces, development setup, production maintenance, etc. Check out
all the markdown documents in this directory tree for information organized by topic.
Development / Setup
-------------------
_develop/docs/local_development/general_setup.md
+127 -60
View File
@@ -10,69 +10,150 @@ The applications (binary targets) you might be interested in running include:
- `download-job`, which downloads models from the web and stores them in GCS
- `inference-job`, which downloads models from GCS and executes them
Database Setup
--------------
Linux Setup (tested on Ubuntu 22.04)
------------------------------------
### Linux Install
1) Install Rust [using this guide](https://www.rust-lang.org/learn/get-started).
Install the following libraries, and see the notes further below about MySQL on Ubuntu 20.04.
2) Install the following libraries and MySQL server:
```bash
jq # for combining mysql codegen outputs
libmysqlclient-dev
libsqlite3-dev
mysql-server
pkgconf # if using openssl instead of rustls
```
```bash
sudo apt install \
jq \
libmysqlclient-dev \
libsqlite3-dev \
mysql-server \
pkgconf
```
### Mac Install
(Note: jq is for combining mysql codegen outputs; pkgconf is if using openssl instead of rustls.)
```bash
brew install mysql
```
3) Install a `storyteller` user and table
If mysql -uroot fails, reboot the machine:
Connect to mysql:
```bash
sudo reboot now
```
```bash
# If the following command asks for a password, the password is typically "root"
sudo mysql -u root -p
```
### Database Migrations (Install Tooling)
Once in MySQL, run the following:
To manage the database and perform migrations, install the Rust tools diesel and sqlx.
```mysql
use mysql;
CREATE DATABASE storyteller;
CREATE USER 'storyteller'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON storyteller.* TO 'storyteller'@'localhost';
```
We'll be using diesel to manage the migrations, but sqlx within the app to actually perform queries.
Diesel is an immature ORM, which isn't a good tech bet, so we use sqlx as at-compile-time
typesafe SQL.
Then verify access with `./dev_mysql_connect.sh`
Install the SQLx CLI (Linux + Mac):
4) Install Diesel CLI (for MySQL migrations):
```bash
cargo install sqlx-cli --no-default-features --features rustls,mysql,sqlite
```
```bash
cargo install diesel_cli \
--no-default-features \
--features mysql,sqlite
```
Install the Diesel CLI (Linux + Mac):
5) Run the pending database migrations:
```bash
cargo install diesel_cli \
--no-default-features \
--features mysql,sqlite
```
```bash
diesel migration run
```
You might get a scary message about `"Encountered unknown type for Mysql: enum"` -- you can ignore this
error (see below).
Mac [has issues](https://github.com/diesel-rs/diesel/issues/2605) and requires a few dependencies to be installed:
```bash
brew install libpq
```
6) (optional) Install the SQLx CLI (if doing database development)
### Database Migrations (Run)
```bash
cargo install sqlx-cli --no-default-features --features rustls,mysql,sqlite
```
To run pending migrations, execute the following:
We'll be using diesel to manage the migrations, but sqlx within the app to actually perform queries.
Diesel is an immature ORM, which isn't a good tech bet, so we use sqlx as at-compile-time
typesafe SQL.
```bash
diesel migration run
```
Mac Setup (tested on Apple M2 Silicon)
--------------------------------------
1) Install Rust [using this guide](https://www.rust-lang.org/learn/get-started).
2) Install MySQL server:
```bash
brew install mysql
```
If `mysql -uroot` fails, reboot the machine:
```bash
sudo reboot now
```
3) Install a `storyteller` user and table
Connect to mysql:
```bash
# If the following command asks for a password, the password is typically "root"
sudo mysql -u root -p
```
Once in MySQL, run the following:
```mysql
use mysql;
CREATE DATABASE storyteller;
CREATE USER 'storyteller'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON storyteller.* TO 'storyteller'@'localhost';
```
Then verify access with `./dev_mysql_connect.sh`
4) Install Diesel CLI (for MySQL migrations):
```bash
cargo install diesel_cli \
--no-default-features \
--features mysql,sqlite
```
As of this writing, Mac [has some issues with Diesel CLI](https://github.com/diesel-rs/diesel/issues/2605)
and requires a few dependencies to be installed:
```bash
brew install libpq
```
5) Run the pending database migrations:
```bash
diesel migration run
```
You might get a scary message about `"Encountered unknown type for Mysql: enum"` -- you can ignore this
error (see below).
6) (optional) Install the SQLx CLI (if doing database development)
```bash
cargo install sqlx-cli --no-default-features --features rustls,mysql,sqlite
```
We'll be using diesel to manage the migrations, but sqlx within the app to actually perform queries.
Diesel is an immature ORM, which isn't a good tech bet, so we use sqlx as at-compile-time
typesafe SQL.
Windows Setup
-------------
(TBD)
Solutions to Common Setup Problems
-----------------------------------
### [Fix] Sqlx Error on Linux when Performing Schema Migrations
@@ -81,28 +162,14 @@ You might get this error message during migration,
```
Encountered unknown type for Mysql: enum
thread 'main' panicked at 'internal error: entered unreachable code: Mysql only supports a closed set of types.
If you ever see this error message please open an issue at https://github.com/diesel-rs/diesel containing a dump of your schema definition.', diesel_cli/src/print_schema.rs:310:17
If you ever see this error message please open an issue at https://github.com/diesel-rs/diesel
containing a dump of your schema definition.', diesel_cli/src/print_schema.rs:310:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
I haven't found the cause (it doesn't happen on Mac), but the migrations appear to work regardless
I haven't found the cause (it doesn't happen on newer installs), but the migrations appear to work regardless
of this error message. You can essentially ignore it.
### [Fix] Can't connect to local MySQL after install on Ubuntu
- If MySql in local dev can't be connected to, [reset the accounts](https://linuxconfig.org/how-to-reset-root-mysql-mariadb-password-on-ubuntu-20-04-focal-fossa-linux).
### [Fix] MySql on Ubuntu 22.04
Should work largely out of the box. We'll need a dev account:
```bash
# Connect to mysql:
# sudo mysql -u root -p (password is "root")
use mysql;
CREATE USER 'storyteller'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON storyteller.* TO 'storyteller'@'localhost';
```
Then verify with `./dev_mysql_connect.sh`