{"id":150,"date":"2021-05-18T20:45:10","date_gmt":"2021-05-18T20:45:10","guid":{"rendered":"https:\/\/agileleaf.com\/blog\/set-up-gitlab-for-continous-deployments\/"},"modified":"2024-04-13T12:23:25","modified_gmt":"2024-04-13T07:23:25","slug":"set-up-gitlab-for-continous-deployments","status":"publish","type":"post","link":"https:\/\/agileleaf.com\/blog\/set-up-gitlab-for-continous-deployments\/","title":{"rendered":"Set up GitLab for Continuous Deployments"},"content":{"rendered":"
Summary:<\/h2>\n
This article lays out the steps needed for setting up Continuous Deployment(CD) from a GitLab Repository to a Ubuntu server. You will need basic knowledge about servers and GitLab to keep up. For me personally, I was linking a simple Laravel web app repo to the deployment server using the following technology stack<\/p>\n
\n
Ubuntu Server with SSH access<\/li>\n
Apache2<\/li>\n
MySQL for Database<\/li>\n
GitLab Repository<\/li>\n<\/ul>\n
Briefly, I followed the below steps for a successful Continuous Deployment.<\/p>\n
\n
Configuring SSH access to GitLab Repository from Ubuntu Server without password (using OpenSSH Private\/Public Keys)<\/li>\n
This allows the server to pull changes when new commits are pushed<\/li>\n
Cloning GitLab repository or Adding Remote Repository<\/li>\n
Configuring SSH access to Ubuntu Server from GitLab Repository without password (using OpenSSH Private\/Public Keys)<\/li>\n
This allows GitLab to access Ubuntu server and run commands<\/li>\n
Making the .gitlab-ci.yml to define Pipeline for CD. (Or steps for GitLab to follow when commits are pushed)<\/li>\n<\/ul>\n
Background:<\/h2>\n
I normally work on small scale projects as a solo developer, so using Git extensively with Continuous-Integration\/Continuous-Deployment (CI\/CD) capabilities didn\u2019t seem much beneficial in my workflow; Plus with no formal training or experience working in big teams, I never really got into the collaboration capabilities made available by Git.<\/p>\n
Recently I started a project on which I had to collaborate, so I thought this was a good opportunity to get my feet wet on Continuous Deployment. The project was a simple Admin Panel for a mobile application and no tests were written. The main goal was automatic deployment to my server whenever a commit was pushed to my GitLab repository.<\/p>\n
I did not require Testing, Continuous Integration or Docker Deployment. While there are many guides on how to do CI\/CD with varying levels of feature implementation, I wanted a bare-bones solution with minimal complexity and time required. As a total newbie in GitLab CI\/CD, with only basic knowledge of Linux, and most of the guides revolving around Docker deployments, I had some trouble figuring out how to implement Continuous Deployment.<\/p>\n
So, this is an article for someone looking for a basic deployment configuration with GitLab with a Ubuntu Server, and for me as future reference.<\/p>\n
GitLab Repository Access from Server using SSH<\/h2>\n
GitLab repository will be configured as such that it can be accessed via SSH without being prompted for a password. This way our server will be able to pull the latest changes from the repo via simple git pull<\/strong> command.<\/p>\n
We will go through these steps<\/p>\n
\n
Create or Find a Public\/Private key pair<\/li>\n
Copy the public key to GitLab Repository<\/li>\n
Configure GitLab on Ubuntu Server to use the private key when accessing the GitLab Repository<\/li>\n<\/ul>\n
Create or Find a Public\/Private key pair:<\/strong><\/p>\n
For this we will SSH into the Ubuntu Server to find the already generated keys or make new ones.<\/p>\n
Once connected to your server open the command line and go to the .ssh<\/strong> folder. The following command can be used.<\/p>\n
cd ~\/.ssh<\/code><\/pre>\n
Once in the directory see if you have any key pairs available.<\/p>\n
ls -a<\/code><\/pre>\n
Generally, you would be able to find two files there name \u201cid_rsa.pub<\/strong>\u201d(Public Key) and \u201cid_rsa<\/strong>\u201d(Private Key)<\/p>\n
In case the key pairs were not found. Use the following command to generate SSH keys<\/p>\n
ssh-keygen -t rsa<\/code><\/pre>\n
When prompted for a passphrase, just press enter to keep it empty.<\/p>\nssh-keygen -t rsa<\/figcaption><\/figure>\n
Afterwards, the key pairs will be available in the directory you specified during key creation.<\/p>\n
Copy the public key to GitLab Repository<\/strong><\/p>\n
Now go to the directory where the keys were generated. Open the id_rsa.pub<\/strong>(Public Key) with any text editor and copy its content. It will look similar to this<\/p>\n\n
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQt2SKv\/H\/SGHy1VgA8mfFG+0WdZexBO24AycNilAQEJkTPkvxkAq42\/am4X9JZl\/TjCvEUKCYXeays3fq1ZD6TIJeeOhOf0YqPsrQnyC98cFEM\/f5v63oWp6cO8h+tynB7TZoT18ItwIOgWKvXO9GmUdUu7CrTAF8b2iC0r08RWxopYEQD4uWFy8iR9wmasdaWe8c\/KyeO\/GjqX+2+jmNIKe\/p8sjnPqN+ArbKeYdI3nI5GdzbfUg7Po471m4cCcWNTeUpVPxrLcwWMjC85P3FPm4BzOX2l2LU7C9+Ln+MzJaTDWBjMb5u2zSesLyx1m91QNDo0Kf1f9pMasdasdaE84C\/\/FkL0ZCiufEwmZjXRguBH+e\/SQ\/A+lM9bHu7HN14tqHL6xGGozObc5tXfbkuTeP0CAGbUT35yRPkckbxblf5wGN5eiQQIjvF6JbBP9bRwvHZUW2BbgmH4hG4HTTyXRj\/UvN1EoEtccQZYqxG9pFb3o9OxC1kk= abid1208@MSI<\/code><\/pre>OpenSSH public key<\/figcaption><\/figure>\n
Login in to your GitLab account and go to your Repository. Navigate to Your-Project<\/strong> \u21fe Settings <\/strong>\u21fe Repository <\/strong>\u21fe Deploy keys<\/strong>.<\/p>\nDeploy Key GitLab<\/figcaption><\/figure>\n
Give a Title<\/strong> to your key, it can be any text which you can later identify. Then paste the contents of id_rsa.pub<\/strong> in the Key <\/strong>field and save.<\/p>\n
Note: In the case you will be deploying several web apps to the same server, and you have previously added the SSH key to that server for another repo; That key will be shown in \u201cPrivately accessible deploy keys\u201d tab, and you can just \u201c<\/em>Enable<\/em><\/strong>\u201d that key for this repository.<\/em><\/p><\/blockquote>\nPreviously added deploy keys<\/figcaption><\/figure>\n
Configure GitLab on Ubuntu Server to use the private key when accessing the GitLab Repository<\/strong> \nNext we need to configure our Ubuntu server to use the private key we created(or was already available) whenever we try to SSH to GitLab. To do this we need to ensure that ssh-agent <\/strong>is enabled by using the following command in the terminal<\/p>\n
eval $(ssh-agent -s)<\/code><\/pre>\n
Then we can add the private key to the SSH Registry by<\/p>\n
# Add your private keyssh-add ~\/.ssh\/id_rsa<\/code><\/pre>\n
Be sure to use the directory in which the key is placed instead of \u2018~\/.ssh<\/strong>\u2019<\/p>\n
We also need to save these settings in a configuration file. Usually OpenSSH clients have a config file in the ~\/.ssh\/<\/strong> directory under the name of config<\/strong>. If this file is not available, you can just create it using<\/p>\n
touch config<\/code><\/pre>\n
in your .ssh<\/strong> directory.<\/p>\n
Note: if your .ssh directory is under \/home\/user_name<\/strong>, you will need to create the config file there.<\/em><\/p><\/blockquote>\n
Open the config file and add the following lines to it via a text editor<\/p>\n\n
#GitLab.com serverHost gitlab.comRSAAuthentication yes#Be sure to replace the ~\/.ssh with the directory in which private key is storedIdentityFile ~\/.ssh\/id_rsa<\/code><\/pre>ssh config file<\/figcaption><\/figure>\n
In case you have private server for different projects you can use multiple key pairs and configure them to use separate keys<\/p>\n\n
Now we can access our GitLab repo via SSH without password prompts. You can verify you SSH configuration by running the following command<\/p>\n
ssh -T git@gitlab.com<\/code><\/pre>\n
You should see a welcome message<\/p>\n
Welcome to GitLab, @username<\/code><\/pre>\n
Cloning GitLab repository<\/h2>\n
We will now clone our GitLab repository to the Server. \nFirst login to your GitLab account and navigate to the relevant project.<\/p>\n
Click the clone button and then copy the SSH link under \u201cClone with SSH<\/strong>\u201d. We will use this link to clone the Repository on our server.<\/p>\nClone with SSH on GitLab<\/figcaption><\/figure>\n
Now login to your Server via SSH and navigate to the web app directory. If you are using Apache2, it will usually be under \u201c\/var\/www\/html\/project-folder\/<\/strong>\u201d<\/p>\n
Note if you are using custom configuration or other server than Apache2, the directory for your web apps will be different.<\/p><\/blockquote>\n
When in the directory run the below command.<\/p>\n
Where you will replace “git@gitlab.com:user-name\/project.git”<\/strong> with the link you copied from your project repository.<\/p>\n
After the project has been cloned you can run the following command in the project directory to confirm that you have access to the repository and it is pointed correctly.<\/p>\n
git ls-remote<\/code><\/pre>\n
If everything is correctly configured you should see something like this<\/p>\ngit ls-remote correct message<\/figcaption><\/figure>\n
If you previously cloned the repository using HTTP, you can change it to SSH by running the following commands.<\/p>\n
Where you will replace the ‘git@gitlab.com:username\/projectpath.git<\/strong>‘ with your project SSH link.<\/p>\n
In my case, I faced a bizarre problem, when I was updating my remote URL from HTTP to SSH, the commands did not seem to change the git config file and I had to manually change its URL. If you are facing connection and permission issues with your repo, checking this file out might help. You can locate this file in “project_folder\/.git\/config<\/strong>“<\/p><\/blockquote>\n
Configuring SSH access to Ubuntu Server from GitLab Repository<\/h2>\n
To run commands on our server, GitLab needs SSH access. For this article we will be using Shared GitLab runners. For more security and control over runners, you can install a Specific Runner on your own Server as well.<\/p>\n
Extra Info: Runners are responsible for executing the commands specified by us upon every new commit. There are several types(executors) of runners available, but we are using the default Docker one. Essentially, as I understand it, whenever a commit is pushed, the runner loads a Docker image, in our case an Alpine Linux distribution and runs the commands specified. All these configs and commands are written in the gitlab-ci.yml file in our project repo, which we will look at in the next section.<\/em><\/p><\/blockquote>\n
Remember the SSH keys we generated in the first step. We will now store that Key(private one) in a GitLab variable. This allows GitLab(runner) to use that key to SSH into our server and run commands. \nIf you were following the steps and are using Ubuntu server, your private key will be in “~\/.ssh\/id_rsa”<\/strong> where id_rsa<\/strong> is the private key. Open the key in a text editor and copy its content. The content will be similar to the one shown below.<\/p>\n
Next, we will navigate to our GitLab Repository, Settings<\/strong> \u21fe CI\/CD<\/strong> \nThen in the variables section we will “Add a variable<\/strong>“.<\/p>\nAdd variable GitLab CI\/CD<\/figcaption><\/figure>\n
Then in the \u201cKey\u201d field, give a name to the variable. I will be using SSH_PRIVATE_KEY. Remember this variable will be referenced in other files and the Key field will be used<\/em>.<\/p>\n
In the \u201cValue\u201d field, paste the contents of the private SSH key and click the \u201cAdd variable<\/strong>\u201d button to save.<\/p>\n
Making the gitlab-ci.yml for defining Pipeline for CD<\/h2>\n
Now that we have set up all the keys and configs, our server and GitLab can communicate with each other. Our gitlab-ci.yml file contains all the instructions for GitLab(runner) on what to do when a new commit is pushed, this is called a Pipeline. \nBelow is the code for my gitlab-ci.yml file. In short, it deploys Docker with Linux image, installs OpenSSH, adds the SSH_PRIVATE_KEY variable we stored in the previous step, and then connects to our Server to run the commands.<\/p>\n\n
The \u201cimage: alpine<\/strong>\u201d is specifying which docker image to use. In our case it is Alpine, which is a very lightweight distribution of Linux.<\/p>\n
Next we specify the stages, a way to distribute our pipeline. We can have multiple stages which can be run sequentially or in parallel. We only have one stage of \u201cdeployment\u201d.<\/p>\n
Now we define what to do in the stages. We first specify what stage it is<\/p>\n
deploy:stage: deployment<\/code><\/pre>\n
Then we define what commands to run in before_script<\/strong> and script<\/strong>. Commands under before_script<\/strong> runs before anything else, and here we are installing and configuring SSH Client which will allow our Docker container with Linux to connect to our server.<\/p>\n
In script<\/strong> we SSH into our Ubuntu server to run commands. You will have to replace the IP with the IP of your server.<\/p>\n
Then we run the following commands<\/p>\n
\n
Go to the project directory<\/li>\n
Specify the branch of our GitLab Repository<\/li>\n
Pull the latest code from Project Repository<\/li>\n
Composer dump-autoload to ensure Laravel finds all new migration files<\/li>\n
Run the migrations to make changes to Database, if any<\/li>\n
Exit<\/li>\n<\/ol>\n
Note that commands 4 & 5 are only applicable to Laravel applications. You can run any Linux commands here specific to your needs because we are using a Ubuntu server. We can also run commands from a file to make our gitlab-ci.yml cleaner.<\/p>\n
At the end we specify \u201conly: master<\/strong>\u201d so that the scripts will only run if the current branch is master.<\/p>\n
Conclusion<\/h2>\n
So this is it. Now our GitLab repository and server are connected and every new commit should be automatically deployed to the Server. Our CI file can be modified to run any command on our server. \nThis is just a basic deployment Pipeline and you can read further in the GitLab documentation for complex scenarios including testing and integration.<\/p>\n
![\"\"](\"https:\/\/lh6.googleusercontent.com\/jnHfOVVBXUVKu5wzu6Wu2HgmYxUqKjYJFynriQJKGm6vAz2mjG3j8r1hNdSWqyyueI4Odytsw3w7l2lORFVfDGYNWHTh1jZWL2bzSZoAUoqcxitGMkIiIdeMxCNK6qcNTVlTHWjy\")
![\"\"](\"https:\/\/lh5.googleusercontent.com\/ge9zOVn4pNgNpVATH_dy5TB3idxcB3D6vhoLFItAyyfrY94Z7So-fodlzgxxzMK0X_rKm3JI-ihy_dtyjFwbe67gH3rB6Aifjws0lPgkFMiZ4Pqb9cGNC28jSNLf_EPMpUcFK7ad\")
![\"\"](\"https:\/\/lh4.googleusercontent.com\/6rnF1MtICWgB359qQpFakT1Ife4kEdH_cuZcH0TfKpSofoRCszzWSF_EQrmRU6BwAwa5_HuJTzf525LfsMD9RVrMNI5HimfPyj8mcAy8TPY2CqvaJTfeWA4eVs1ISBCNtWkD8iEE\")
![\"\"](\"https:\/\/lh3.googleusercontent.com\/HvBMnhIUGicBD8F7MdoR063mHGR6VcsWL0qBdOCl3jSDouTCobHDTxBKq2CHZakEAaRNTwQuEHmmP4BebXVScinL7qdsjlKABmGg4rxIr7Y8Zq679dro4gVLxITcPvD70Q-fh5a8\")
![\"\"](\"https:\/\/lh6.googleusercontent.com\/-CZxBjwy_1AdMd_OLlRq291vr9Dt2YQXqo_VTcbzHYpsT1-co46Kj_1TvK7l2hzrndSnyWFU0ukAV9U7Kgknf333gWwIqNt1iL551Q6l6LVOsBXaL7IMAgFTY-y4m6FTeEOvuM_4\")
![\"\"](\"https:\/\/lh6.googleusercontent.com\/HaozRBFA8NWsY64KhMOrUaE9Cdcw11znSr-pfldW8jXlJcBX9VRvKwL7FhtVxrNV9nlyDr3BMEhJ7WQ5xWKZPsPcaqYoCo6lMKhiJTdYYXai3haUh15zwEGnijedtmuEhpHk7qBu\")