Merge remote-tracking branch 'upstream/pull/5310'

[rails.git] / INSTALL.md

diff --git a/INSTALL.md b/INSTALL.md
index 641d1160a1ac9af2df6cc2c27b1fd655977ccce0..8667fb5129161efca38295559202e64fd86475b4 100644 (file)
--- a/INSTALL.md
+++ b/INSTALL.md
@@ -1,14 +1,18 @@
 # Installation
 
 # Installation
 

-These instructions are designed for setting up The Rails Port for development and testing.
+These instructions are designed for setting up `openstreetmap-website` for development and testing.

 If you want to deploy the software for your own project, then see the notes at the end.
 
 If you want to deploy the software for your own project, then see the notes at the end.
 

-You can install the software directly on your machine, which is the traditional and probably best-supported approach. However, there is an alternative which may be easier: Vagrant. This installs the software into a virtual machine, which makes it easier to get a consistent development environment and may avoid installation difficulties. For Vagrant instructions, see [VAGRANT.md](VAGRANT.md).
+You can install the software directly on your machine, which is the traditional and probably best-supported approach. However, there
+are two alternatives which make it easier to get a consistent development environment and may avoid installation difficulties:

 
 

-These instructions are based on Ubuntu 18.04 LTS, which is the platform used by the OSMF servers.
+* **Vagrant** This installs the software into a virtual machine. For Vagrant instructions see [VAGRANT.md](VAGRANT.md).
+* **Docker** This installs the software using containerization. For Docker instructions see [DOCKER.md](DOCKER.md).
+
+These instructions are based on Ubuntu 24.04 LTS, which is the platform used by the OSMF servers.

 The instructions also work, with only minor amendments, for all other current Ubuntu releases, Fedora and MacOSX
 
 The instructions also work, with only minor amendments, for all other current Ubuntu releases, Fedora and MacOSX
 

-We don't recommend attempting to develop or deploy this software on Windows. If you need to use Windows, then try developing this software using Ubuntu in a virtual machine, or use [Vagrant](VAGRANT.md).
+We don't recommend attempting to develop or deploy this software on Windows. Some Ruby gems may not be supported. If you need to use Windows the easiest solutions in order are [Docker](DOCKER.md), [Vagrant](VAGRANT.md), and Ubuntu in a virtual machine.

 
 ## Dependencies
 
 
 ## Dependencies
 


@@ -18,22 +22,21 @@ of packages required before you can get the various gems installed.
 
 ## Minimum requirements
 
 
 ## Minimum requirements
 

-* Ruby 2.5+
-* PostgreSQL 9.1+
-* ImageMagick
-* Bundler
+* Ruby 3.1+
+* PostgreSQL 13+
+* Bundler (see note below about [developer Ruby setup](#rbenv))

 * Javascript Runtime
 
 * Javascript Runtime
 

-These can be installed on Ubuntu 18.04 or later with:
+These can be installed on Ubuntu 24.04 or later with:

 
 ```
 sudo apt-get update
 
 ```
 sudo apt-get update

-sudo apt-get install ruby2.5 libruby2.5 ruby2.5-dev bundler \
-                     libmagickwand-dev libxml2-dev libxslt1-dev nodejs \
-                     apache2 apache2-dev build-essential git-core phantomjs \
+sudo apt-get install ruby ruby-dev ruby-bundler \
+                     libvips-dev libxml2-dev libxslt1-dev nodejs \
+                     build-essential git-core \

                      postgresql postgresql-contrib libpq-dev libsasl2-dev \
                      postgresql postgresql-contrib libpq-dev libsasl2-dev \

-                     imagemagick libffi-dev libgd-dev libarchive-dev libbz2-dev
-sudo gem2.5 install bundler
+                     libffi-dev libgd-dev libarchive-dev libyaml-dev libbz2-dev npm
+sudo npm install --global yarn

 ```
 
 ### Alternative platforms
 ```
 
 ### Alternative platforms


@@ -44,11 +47,11 @@ For Fedora, you can install the minimum requirements with:
 
 ```
 sudo dnf install ruby ruby-devel rubygem-rdoc rubygem-bundler rubygems \
 
 ```
 sudo dnf install ruby ruby-devel rubygem-rdoc rubygem-bundler rubygems \

-                 libxml2-devel js \
+                 libxml2-devel nodejs \

                  gcc gcc-c++ git \
                  gcc gcc-c++ git \

-                 postgresql postgresql-server postgresql-contrib \
-                 perl-podlators ImageMagick libffi-devel gd-devel libarchive-devel \
-                 bzip2-devel nodejs-yarn
+                 postgresql postgresql-server postgresql-contrib libpq-devel \
+                 perl-podlators libffi-devel gd-devel libarchive-devel \
+                 libyaml-devel bzip2-devel nodejs-yarn vips-devel

 ```
 
 If you didn't already have PostgreSQL installed then create a PostgreSQL instance and start the server:
 ```
 
 If you didn't already have PostgreSQL installed then create a PostgreSQL instance and start the server:


@@ -71,6 +74,7 @@ For MacOSX, you will need XCode installed from the Mac App Store; OS X 10.7 (Lio
 Installing PostgreSQL:
 
 * Install Postgres.app from https://postgresapp.com/
 Installing PostgreSQL:
 
 * Install Postgres.app from https://postgresapp.com/

+* Make sure that you've initialized and started Postgresql from the app (there should be a little elephant icon in your systray).

 * Add PostgreSQL to your path, by editing your profile:
 
 `nano ~/.profile`
 * Add PostgreSQL to your path, by editing your profile:
 
 `nano ~/.profile`


@@ -79,14 +83,26 @@ and adding:
 
 `export PATH=/Applications/Postgres.app/Contents/MacOS/bin:$PATH`
 
 
 `export PATH=/Applications/Postgres.app/Contents/MacOS/bin:$PATH`
 

+After this, you may need to start a new shell window, or source the profile again by running `. ~/.profile`.
+

 Installing other dependencies:
 
 * Install Homebrew from https://brew.sh/
 * Install the latest version of Ruby: `brew install ruby`
 Installing other dependencies:
 
 * Install Homebrew from https://brew.sh/
 * Install the latest version of Ruby: `brew install ruby`

-* Install ImageMagick: `brew install imagemagick`
-* Install libxml2: `brew install libxml2 --with-xml2-config`
-* If you want to run the tests, you need `phantomjs` as well: `brew install phantomjs`
-* Install Bundler: `gem install bundler`
+* Install other dependencies: `brew install libxml2 gd yarn pngcrush optipng pngquant jhead jpegoptim gifsicle svgo advancecomp vips`
+* Install Bundler: `gem install bundler` (you might need to `sudo gem install bundler` if you get an error about permissions - or see note below about [developer Ruby setup](#rbenv))
+
+You will need to tell `bundler` that `libxml2` is installed in a Homebrew location. If it uses the system-installed one then you will get errors installing the `libxml-ruby` gem later on<a name="macosx-bundle-config"></a>.
+
+```
+bundle config build.libxml-ruby --with-xml2-config=/usr/local/opt/libxml2/bin/xml2-config
+```
+
+If you want to run the tests, you need `geckodriver` as well:
+
+```
+brew install geckodriver
+```

 
 Note that OS X does not have a /home directory by default, so if you are using the GPX functions, you will need to change the directories specified in config/application.yml.
 
 
 Note that OS X does not have a /home directory by default, so if you are using the GPX functions, you will need to change the directories specified in config/application.yml.
 


@@ -109,7 +125,7 @@ If you want to add in the full history later on, perhaps to run `git blame` or `
 
 ## Ruby gems
 
 
 ## Ruby gems
 

-We use [Bundler](http://gembundler.com/) to manage the rubygems required for the project.
+We use [Bundler](https://bundler.io/) to manage the rubygems required for the project.

 
 ```
 cd openstreetmap-website
 
 ```
 cd openstreetmap-website


@@ -121,12 +137,20 @@ bundle install
 We use [Yarn](https://yarnpkg.com/) to manage the Node.js modules required for the project.
 
 ```
 We use [Yarn](https://yarnpkg.com/) to manage the Node.js modules required for the project.
 
 ```

-bundle exec rake yarn:install
+bundle exec bin/yarn install
+```
+
+## Prepare local settings file
+
+This is a workaround. [See issues/2185 for details](https://github.com/openstreetmap/openstreetmap-website/issues/2185#issuecomment-508676026).
+
+```
+touch config/settings.local.yml

 ```
 
 ## Storage setup
 
 ```
 
 ## Storage setup
 

-The Rails port needs to be configured with an object storage facility - for
+`openstreetmap-website` needs to be configured with an object storage facility - for

 development and testing purposes you can use the example configuration:
 
 ```
 development and testing purposes you can use the example configuration:
 
 ```


@@ -135,7 +159,7 @@ cp config/example.storage.yml config/storage.yml
 
 ## Database setup
 
 
 ## Database setup
 

-The Rails Port uses three databases -  one for development, one for testing, and one for production. The database-specific configuration
+`openstreetmap-website` uses three databases -  one for development, one for testing, and one for production. The database-specific configuration

 options are stored in `config/database.yml`, which we need to create from the example template.
 
 ```
 options are stored in `config/database.yml`, which we need to create from the example template.
 
 ```


@@ -161,25 +185,7 @@ exit
 To create the three databases - for development, testing and production - run:
 
 ```
 To create the three databases - for development, testing and production - run:
 
 ```

-bundle exec rake db:create
-```
-
-### PostgreSQL Btree-gist Extension
-
-We need to load the `btree-gist` extension, which is needed for showing changesets on the history tab.
-
-```
-psql -d openstreetmap -c "CREATE EXTENSION btree_gist"
-```
-
-### PostgreSQL Functions
-
-There are special database functions required by a (little-used) API call, the migrations and diff replication. The former two are provided as *either* pure SQL functions or a compiled shared library. It is recommended that you start with the pure SQL versions, as described below, and only install the compiled shared library if you are running a production server making a lot of `/changes` API calls or need the diff replication functionality.
-
-If you aren't sure which you need, install the SQL version below.
-
-```
-psql -d openstreetmap -f db/functions/functions.sql
+bundle exec rails db:create

 ```
 
 ### Database structure
 ```
 
 ### Database structure


@@ -187,7 +193,7 @@ psql -d openstreetmap -f db/functions/functions.sql
 To create all the tables, indexes and constraints, run:
 
 ```
 To create all the tables, indexes and constraints, run:
 
 ```

-bundle exec rake db:migrate
+bundle exec rails db:migrate

 ```
 
 ## Running the tests
 ```
 
 ## Running the tests


@@ -195,7 +201,7 @@ bundle exec rake db:migrate
 To ensure that everything is set up properly, you should now run:
 
 ```
 To ensure that everything is set up properly, you should now run:
 
 ```

-bundle exec rake test:db
+bundle exec rails test:all

 ```
 
 This test will take a few minutes, reporting tests run, assertions, and any errors. If you receive no errors, then your installation is successful.
 ```
 
 This test will take a few minutes, reporting tests run, assertions, and any errors. If you receive no errors, then your installation is successful.


@@ -218,41 +224,26 @@ Note that the OSM map tiles you see aren't created from your local database - th
 
 After installing this software, you may need to carry out some [configuration steps](CONFIGURE.md), depending on your tasks.
 
 
 After installing this software, you may need to carry out some [configuration steps](CONFIGURE.md), depending on your tasks.
 

-# Installing compiled shared library database functions
+# Ruby development install and versions<a name="rbenv"></a> (optional)

 
 

-You probably only need to do this if you are running a large, production instance of openstreetmap-website.
+For simplicity, this document explains how to install all the website dependencies as "system" dependencies. While this is simpler, and usually faster, you might want more control over the process or the ability to install multiple different versions of software alongside each other. For many developers, [`rbenv`](https://github.com/rbenv/rbenv) is the easiest way to manage multiple different Ruby versions on the same computer - with the added advantage that the installs are all in your home directory, so you don't need administrator permissions.

 
 

-Before installing the functions, it's necessary to install the PostgreSQL server development packages. On Ubuntu this means:
+If you choose to install Ruby and Bundler via `rbenv`, then you do not need to install the system libraries for Ruby:

 
 

-```
-sudo apt-get install postgresql-server-dev-all
-```
-
-On Fedora:
-
-```
-sudo dnf install postgresql-devel
-```
+* For Ubuntu, you do not need to install the following packages: `ruby ruby-dev ruby-bundler`,
+* For Fedora, you do not need to install the following packages: `ruby ruby-devel rubygem-rdoc rubygem-bundler rubygems`
+* For MacOSX, you do not need to `brew install ruby` - but make sure you've installed a version of Ruby using `rbenv` before running `gem install bundler`!

 
 

-The library then needs compiling.
+After installing a version of Ruby with `rbenv` (the latest stable version is a good place to start), you will need to make that the default. From inside the `openstreetmap-website` directory, run:

 
 ```
 
 ```

-cd db/functions
-make libpgosm.so
-cd ../..
+rbenv local $VERSION

 ```
 
 ```
 

-If you previously installed the SQL versions of these functions, we'll need to delete those before adding the new ones:
+Where `$VERSION` is the version you installed. Then install bundler:

 
 ```
 
 ```

-psql -d openstreetmap -c "DROP FUNCTION IF EXISTS maptile_for_point"
-psql -d openstreetmap -c "DROP FUNCTION IF EXISTS tile_for_point"
+gem install bundler

 ```
 
 ```
 

-Then we create the functions within each database. We're using `pwd` to substitute in the current working directory, since PostgreSQL needs the full path.
-
-```
-psql -d openstreetmap -c "CREATE FUNCTION maptile_for_point(int8, int8, int4) RETURNS int4 AS '`pwd`/db/functions/libpgosm', 'maptile_for_point' LANGUAGE C STRICT"
-psql -d openstreetmap -c "CREATE FUNCTION tile_for_point(int4, int4) RETURNS int8 AS '`pwd`/db/functions/libpgosm', 'tile_for_point' LANGUAGE C STRICT"
-psql -d openstreetmap -c "CREATE FUNCTION xid_to_int4(xid) RETURNS int4 AS '`pwd`/db/functions/libpgosm', 'xid_to_int4' LANGUAGE C STRICT"
-```
+You should now be able to proceed with the rest of the installation. If you're on MacOSX, make sure you set up the [config override for the libxml2 location](#macosx-bundle-config) _after_ installing bundler.