Heaving own local repository mirror is handy because you can use single source to update multiple database nodes. You will also save your time and avoid mounting ISO files individually on every node.
Consider these two repositories as minimal set for my Exadata db nodes (example taken from my previous post related do YUM):
[exadata_dbserver_220.127.116.11.3_x86_64_base] – baseline of my current Exadata firmware
[ol6_u8_x86_64_base]– base repository for Oracle Linux 6 Update 8.
Unfortunately first is not available on Public Oracle YUM server. It can be created out of ISO files downloaded from Oracle or mirrored from Unbreakable Linux Network (ULN). Second, generic repository is available on both: ULN and public YUM servers (for example here).
What methods can be used to prepare such local mirror with these two repositories?
uln-yum-mirrorto mirror both repositories from ULN.
- Use ISO files downloaded from MOS for Exadata baseline and Oracle Public Yum server for other packages.
- Use docker container .
First is well documented here and here, however it has one disadvantage: it requires system with Oracle Linux to host such ULN mirror. While it should not be a problem to create small virtual machine for OL – many Red Hat customers seems to have problem with it. They buy Oracle Exadata but they don’t want to use Oracle Linux anywhere else in their environment.
For such a case two other methods might be effective. Until recently – when I dealt with Red-Hat-only-customer – I used to create local mirror out of ISO file downloaded from MOS together with OL base repository mirrored from Oracle public YUM server using reposync (or also from ISO). This can be done on any Linux which have reposync and createrepo installed (even Debian-based system can deal with it). But I couldn’t figure out how to use reposync with ULN on Red Hat or any other non-Oracle linux distribution.
Recently I’ve discovered that I might use oraclelinux docker image on any Linux distribution to register it in the ULN (!) as an YUM server. The only requirement is the system with docker support, for example Red Hat 7 and ULN subscription (which you have if you are Exadata customer).
Setting up docker
It might be a case that you don’t have docker installed. On Red Hat 7 this can be done this way:
[root@rhel ~]# subscription-manager repos --enable=rhel-7-server-rpms [root@rhel ~]# subscription-manager repos --enable=rhel-7-server-extras-rpms [root@rhel ~]# yum install docker [root@rhel ~]# systemctl enable docker [root@rhel ~]# systemctl start docker [root@rhel ~]# docker info
Create directory structure to hold persistenta data and repo files
Here I’m going to use directory on my host which will be mounted as a volume into docker container. This is just easier to start with but later I’ll show how to keep this data inside separate docker volume – which what you should actually do when working with docker.
[root@rhel ~]# mkdir -p /yum/conf [root@rhel ~]# mkdir -p /yum/repos [root@rhel ~]# mkdir -p /yum/persist [root@rhel ~]# mkdir -p /yum/cache [root@rhel ~]# chcon -Rt svirt_sandbox_file_t /yum [root@rhel ~]# cat >/yum/conf/yum.conf <<EOF [main] logfile=/var/log/yum.log gpgcheck=1 plugins=1 pluginconfpath=/yum/conf persistdir=/yum/persist cachedir=/yum/cache EOF cat >/yum/conf/rhnplugin.conf <<EOF [main] enabled=1 gpgcheck=1 EOF cat >/yum/conf/ulninfo.conf <<EOF [main] enabled=1 gpgcheck=1 EOF
FYI: highlighted command is one of the ways to avoid protection SELinux imposes on docker accessing host directory.
Create and start container
Now let’s start container using oraclelinux image. It should be automatically located in official docker repository, downloaded and used to run container called “uln-mirror”:
[root@rhel ~]# docker run -it \ --name "uln-mirror" \ -h "uln-mirror" \ -v /yum:/yum \ oraclelinux
Once started you get the prompt of the bash running inside new container. You should also have /yum directory mounted from host with files created in previous step.
[root@uln-mirror /]# df -h Filesystem Size Used Avail Use% Mounted on overlay 37G 15G 23G 39% / tmpfs 920M 0 920M 0% /dev tmpfs 920M 0 920M 0% /sys/fs/cgroup /dev/mapper/rhel_rhel-root 37G 15G 23G 39% /yum shm 64M 0 64M 0% /dev/shm tmpfs 920M 0 920M 0% /proc/acpi tmpfs 920M 0 920M 0% /proc/scsi tmpfs 920M 0 920M 0% /sys/firmware [root@uln-mirror /]# find /yum /yum /yum/conf /yum/conf/yum.conf /yum/conf/rhnplugin.conf /yum/conf/ulninfo.conf /yum/repos /yum/persist /yum/cache [root@uln-mirror /]# cat /etc/oracle-release Oracle Linux Server release 7.6
Register docker container as YUM server
So, despite the fact that I’m running Red Hat Enterprise Linux 7.6 – I do have an container on it which identifies itself as Oracle Linux 7.6. Moreover
uln_register command is available there, so why not try to register it in ULN. But before executing it, I need to make my registration persistent because everything changed inside container will be lost after its removal. I do this by modifying
/etc/sysconfig/rhn/up2date config file where I change systemIdPath value to /yum/conf/systemid (and this step needs to be done every time new container is created from the image). The
systemid file will be created upon successful registration and If I keep it – I will be able to recreate my setup with another container.
[root@uln-mirror /]# sed -i 's|^systemIdPath=.*$|systemIdPath=/yum/conf/systemid|g' /etc/sysconfig/rhn/up2date [root@uln-mirror /]# uln_register [root@uln-mirror /]# ls -l /yum/conf/systemid -rw-------. 1 root root 1394 06-01 18:19 /yum/conf/systemid
Now, let’s modify my registration so that my container is treated as YUM server (necessary to subscribe any channel from ULN). Also let’s subscribe to repositories I require on my Exadata db nodes:
[root@uln-mirror /]# uln-channel --enable-yum-server [root@uln-mirror /]# uln-channel -a -c exadata_dbserver_18.104.22.168.3_x86_64_base -c ol6_u8_x86_64_base Username: ******* Password: ******* [root@uln-mirror /]# uln-channel -l exadata_dbserver_22.214.171.124.3_x86_64_base ol6_u8_x86_64_base ol7_x86_64_UEKR5 ol7_x86_64_latest
As you can see now I’m subscribed to default OL7 repositories and to these two I explicitly added.
Here is where my
/yum/conf/yum.conf and other files created previously starts to be important. I need to download (differentially if this is consecutive try) repositories into
/yum/repos directory and use persistent cache – so I don’t use default config files from
/etc/yum* (that would be deleted anyway as I’m inside container).
[root@uln-mirror /]# reposync -c /yum/conf/yum.conf -l -m --download-metadata -p /yum/repos -r exadata_dbserver_126.96.36.199.3_x86_64_base -r ol6_u8_x86_64_base ... Loaded plugins: rhnplugin This system is receiving updates from ULN. exadata_dbserver_188.8.131.52.3_x86_64_base | 1.2 kB 00:00:00 exadata_dbserver_184.108.40.206.3_x86_64_base/updateinfo | 153 B 00:00:00 exadata_dbserver_220.127.116.11.3_x86_64_base/primary | 477 kB 00:00:02 ol6_u8_x86_64_base | 1.4 kB 00:00:00 ol6_u8_x86_64_base/group (1/446): Lib_Utils-1.00-09.noarch.rpm | 1.5 MB 00:00:05 (2/446): MAKEDEV-3.24-6.el6.x86_64.rpm | 88 kB 00:00:00 ...
It works and repositories are being downloaded into
/yum/repos. After process finishes the only task left is to create repository out of them:
[root@uln-mirror /]# createrepo -v /yum/repos/exadata_dbserver_18.104.22.168.3_x86_64_base [root@uln-mirror /]# createrepo -v /yum/repos/ol6_u8_x86_64_base
/yum/repos directory using web server, for example Apache as it is described in official documentation.
Better (automated) way
Entire process can be automated and improved, so it could be more aligned to docker philosophy. For example you can use one image for registration and download and another one for publishing repositories locally using HTTP service inside docker container.
For that purpose I created working example of such setup. There is also more information about usage under the link. Below is an example of how it can be used (this is done on my workstation which runs on Ubuntu)
Step 1 – Register
Register creates prepares the image and starts container which will ask you for ULN credentials and CSI number. You only provides docker volume name which will be created (if does not exists yet) or mounted (if you run registration multiple times with the same parameter).
michal@sunman:/tmp$ git clone https://github.com/wes-pro/docker-uln-mirror.git Cloning into 'docker-uln-mirror'... remote: Enumerating objects: 37, done. remote: Counting objects: 100% (37/37), done. remote: Compressing objects: 100% (28/28), done. remote: Total 37 (delta 14), reused 27 (delta 9), pack-reused 0 Unpacking objects: 100% (37/37), done. Checking connectivity... done. michal@sunman:/tmp$ cd docker-uln-mirror michal@sunman:/tmp$ echo exadata_dbserver_22.214.171.124.3_x86_64_base>repo_list michal@sunman:/tmp$ echo ol6_u8_x86_64_base>>repo_list michal@sunman:/tmp/docker-uln-mirror$ ./register.sh uln-vol latest: Pulling from library/oraclelinux a010df13e11b: Pull complete Digest: sha256:f36ae2dff1e4aa8420ca3c3403afa84ab1b77005d164213f5cd02ebd4735ea2c Status: Downloaded newer image for oraclelinux:latest uln-vol Sending build context to Docker daemon 139.8kB Step 1/6 : FROM oraclelinux:latest ---> ccafb8385951 Step 2/6 : RUN yum install -y createrepo ---> Running in 60c3e773f62e Loaded plugins: ovl, ulninfo Resolving Dependencies --> Running transaction check ---> Package createrepo.noarch 0:0.9.9-28.el7 will be installed --> Processing Dependency: python-deltarpm for package: createrepo-0.9.9-28.el7.noarch --> Processing Dependency: deltarpm for package: createrepo-0.9.9-28.el7.noarch --> Running transaction check ---> Package deltarpm.x86_64 0:3.6-3.el7 will be installed ---> Package python-deltarpm.x86_64 0:3.6-3.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: createrepo noarch 0.9.9-28.el7 ol7_latest 93 k Installing for dependencies: deltarpm x86_64 3.6-3.el7 ol7_latest 81 k python-deltarpm x86_64 3.6-3.el7 ol7_latest 30 k Transaction Summary ================================================================================ Install 1 Package (+2 Dependent packages) Total download size: 205 k Installed size: 558 k Downloading packages: -------------------------------------------------------------------------------- Total 377 kB/s | 205 kB 00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : deltarpm-3.6-3.el7.x86_64 1/3 Installing : python-deltarpm-3.6-3.el7.x86_64 2/3 Installing : createrepo-0.9.9-28.el7.noarch 3/3 Verifying : createrepo-0.9.9-28.el7.noarch 1/3 Verifying : deltarpm-3.6-3.el7.x86_64 2/3 Verifying : python-deltarpm-3.6-3.el7.x86_64 3/3 Installed: createrepo.noarch 0:0.9.9-28.el7 Dependency Installed: deltarpm.x86_64 0:3.6-3.el7 python-deltarpm.x86_64 0:3.6-3.el7 Complete! Removing intermediate container 60c3e773f62e ---> dc0d8c389578 Step 3/6 : COPY entrypoint.sh / ---> eadb36308c45 Step 4/6 : COPY repo_list / ---> 36827290e9fa Step 5/6 : ENTRYPOINT ["/entrypoint.sh"] ---> Running in 49c56043df57 Removing intermediate container 49c56043df57 ---> f038f0b3b633 Step 6/6 : CMD ["download"] ---> Running in 4329ea7418ef Removing intermediate container 4329ea7418ef ---> 58041752999c Successfully built 58041752999c Successfully tagged uln-mirror:latest Your ULN username: **** Your ULN password: **** Your CSI: **** Registering your system... Please wait... Successfully registered your system. File: /uln/conf/systemid has been created. List of available repositories has been written to /uln/conf/repo_available Subscribed to repositories: exadata_dbserver_126.96.36.199.3_x86_64_base ol6_u8_x86_64_base Docker image "uln-mirror" and volume "uln-vol" created. If you haven't done so yet, put channel names you want to download into repo_list file and rerun register.sh script with the same volume name. You can find list of all available repositories in repo_available file. Re-running register.sh script with existing volume - still asks for username and password - these are required by uln-channel to modify channel list. Then run below command to start docker and which will download selected repositories: ./download.sh <volume-name> To publish downloaded repositories using HTTP server start another docker using: ./publish.sh <volume-name>
It is possible to execute
register.sh multiple times with the same argument. The script behaves slightly differently when it finds registration data on volume. It only asks for Oracle credentials – but not for CSI – and use them to subscribe channels from
repo_list file. This is for the case when you want to start downloading new channels which were not selected during initial registration.
Step 2 – Download
Download uses the same image and volume and assumes you already registered the system in ULN and registration data is written into
systemid file on the volume. It just download packages from ULN (or update them differentially) and then it recreates local repositories.
michal@sunman:/tmp/docker-uln-mirror$ ./download.sh uln-vol bf32078aab785acdb79c0204439e575f76e266a68cc93d29d12e8b1d8daabf3b michal@sunman:/tmp/docker-uln-mirror$ docker logs -f bf32078aab785acdb79c0204439e575f76e266a68cc93d29d12e8b1d8daabf3b Loaded plugins: ovl, rhnplugin This system is receiving updates from ULN. exadata_dbserver_188.8.131.52.3_x86_64_base | 1.2 kB 00:00 exadata_dbserver_184.108.40.206.3_x86_64_base/updateinfo | 153 B 00:00 exadata_dbserver_220.127.116.11.3_x86_64_base/primary | 477 kB 00:02 ol6_u8_x86_64_base | 1.4 kB 00:00 ol6_u8_x86_64_base/group | 1.2 MB 00:04 ol6_u8_x86_64_base/updateinfo | 1.5 MB 00:05 ...
Step 3 – Publish
Publish does not use oraclelinux docker image. Instead it uses official nginx image which is just web server running inside docker. The container will start publishing downloaded repositories on port 8080.
michal@sunman:/tmp/docker-uln-mirror$ ./publish.sh uln-vol Unable to find image 'nginx:latest' locally latest: Pulling from library/nginx 743f2d6c1f65: Pull complete 6bfc4ec4420a: Pull complete 688a776db95f: Pull complete Digest: sha256:23b4dcdf0d34d4a129755fc6f52e1c6e23bb34ea011b315d87e193033bcd1b68 Status: Downloaded newer image for nginx:latest 1373451bb76aaf6dbda1a610d0f3dd6f4a7a5d77784b4b058f146c01c0a97cec
You can check that last step created docker container with NGINX web server which publishes downloaded repositories on port 8080:
michal@sunman:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1373451bb76a nginx "nginx -g 'daemon of…" 13 hours ago Up 13 hours 0.0.0.0:8080->80/tcp uln-mirror-nginx bf32078aab78 uln-mirror:latest "/entrypoint.sh down…" 13 hours ago Up 13 hours uln-mirror
Check it out using browser:
From now on you can just point your compute nodes to this local repository. For example:
[root@exa01vm03 ~]# cat /etc/yum.repos.d/exadata.repo [exadata_ol6_base_repo_18.104.22.168.3] name=Exadata release 22.214.171.124.3 db server installation packages (x86_64) baseurl=http://yum-mirror:8080/exadata_dbserver_126.96.36.199.3_x86_64_base gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle gpgcheck=1 enabled=1 [ol6_u8_x86_64_base] name=Oracle Linux 6 Update 8 installation media copy (x86_64) baseurl=http://yum-mirror:8080/ol6_u8_x86_64_base gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle gpgcheck=1 enabled=0