Jenkins 503 Error on CentOS 7

My Jenkins install was whining at me to upgrade to the newest release (2.57) for security reasons. I finally relented, and updated the jenkins.war file only to be greeted with this wonderful error:

HTTP ERROR: 503
Problem accessing /. Reason:
    Service Unavailable

Not terribly informative. My first Google perusal revealed that it was probably an error with Jetty, and so I checked jetty configurations. No luck there there though.

I looked at the Jenkins log (/var/log/jenkins/jenkins.log), and found this:

WARNING: Failed to delete the temporary Winstone file /tmp/winstone/jenkins.war
May 01, 2017 12:29:43 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: Logging initialized @923ms
May 01, 2017 12:29:43 PM winstone.Logger logInternal
INFO: Beginning extraction from war file
May 01, 2017 12:29:43 PM org.eclipse.jetty.util.log.JavaUtilLog warn
WARNING: Empty contextPath
May 01, 2017 12:29:43 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: jetty-9.2.z-SNAPSHOT
May 01, 2017 12:29:44 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
May 01, 2017 12:29:44 PM org.eclipse.jetty.util.log.JavaUtilLog warn
WARNING: Failed startup of context w.@18d4479b{/,file:/var/cache/jenkins/war/,STARTING}{/var/cache/jenkins/war}
java.lang.reflect.InvocationTargetException

Looks like I am having trouble writing/removing files. So, I check permissions and users, but nothing looks out of the ordinary there.

I also use my Google-fu to see if I can get any clue on the last warning there. The only similar issues I can find are related to missing font-packs. That doesn’t make too much sense, but I try it any way. No avail.

Finally, I tried upgrading my Java install to 1.8 from 1.7. I looked through the Jenkins release notes, and I didn’t see anything overtly saying that Java 8 is the new requirement, but I did see some notes about Java 9 being supported, and Java 7 being the minimum for slaves.

I followed the instructions here to install Java 8, and what do you know? Success! So there you go. If you upgrade your Jenkins and see this error, try upgrading to Java 8!

 

MediaWiki on CentOS 7

I’ve set up a few installations of MediaWiki in the past on different operating systems. It’s not the type of thing I do a lot, but when I do, It’s usually not too much trouble.

I was migrating an old MediaWiki installation from a Windows 2007 server to CentOS 7. I figured it would be pretty routine. It turns out there were a couple little hiccups along the way that I figure I should document to help me out in the future.

The first problem I ran into is that CentOS 7 doesn’t ship with a new enough version of of PHP for the current release of Media Wiki.

Uninstall built in version of PHP

yum erase php
yum erase php-common

Install yum repo for newer PHP release

rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm

Install PHP packages

yum install php55w* --skip-broken

This allowed MediaWiki to run finally, and I was able to create a new LocalSettings.php. From there I restored the SQL database without any issue. Don’t forget to install MariaDB.

yum install mariadb-server mariadb

To import the old SQL data to the new DB, first creat a DB with the same name, and then pupolate it with the sql file.

# Get into mysql shell
mysql --password=********
# Create DB with same name as old DB
mysql> CREATE DATABASE my_wiki;
# Exit shell

# Import old SQL data to new DB
mysql --password=******** my_wiki < /wiki.sql

I also copied over the images directory from the old installation.

The next issue I had is permissions. SELinux wasn’t even on my mind, and provided all sort of permission problems. I ran the following commands on my html directory where MediaWiki is installed in an attempt to fix everything.

sudo chown apache:apache -R /var/www/html/

find . -type f -exec chmod 0644 {} \;
find . -type d -exec chmod 0755 {} \;

sudo chcon -t httpd_sys_content_t /var/www/html -R
sudo chcon -t httpd_sys_rw_content_t /var/www/html/images/ -R

At this point, MediaWiki is running, but I couldn’t see any images, or upload any files. I tweak settings in LocalSettings.php but to no avail. I enabled logs and better error messages by adding these lines to LocalSettings.php

$wgShowExceptionDetails = true;
$wgDebugLogFile = "/var/log/mediawiki/debug-{$wgDBname}.log";

When I upload a file using MediaWiki, I get the error:

Could not create directory "mwstore://local-backend/local-public/1/1e".

The log file shows the following error:

[FileOperation] mkdir(): Permission denied
[FileOperation] FSFileBackend::doPrepareInternal: cannot create directory /1/1e

I am unable to upload images, so I Google the error message and check my settings. Almost all of the sites talking about this error suggested permission issues with the images folder. I checked my permissions over, and over again with no luck. The rest fo the sites talked about having the $wgTmpDirectory set correctly, which I’m pretty sure I did.

$wgEnableUploads = true;
$wgUseImageMagick = true;
$wgImageMagickConvertCommand = "/usr/bin/convert";
$wgTmpDirectory = '/var/www/html/images/temp';

What bothers me is that the error message in the log looks like it is trying to mkdir at the root level, however MediaWiki switches a lot between url paths (relative) and absolute. It wasn’t clear. The next debugging step was using strace to see what the OS was doing. I attached strace to one of the apache processes and uploaded a file, and lo:

mkdir("/temp", 0777)                    = -1 EACCES (Permission denied)

It is trying to use my root directory as the images fodler location, even though I have a different tmp directory specified in my settings file.

It turns out that there are default setting in the img_auth.php file that don’t appear to be correct. They get automatically set based on the environment. I chose to explicitely set them in the LocalSettings.php document instead.

# Redefining Upload Directory and Path from img_auth.php
$wgUploadDirectory = '/var/www/html/images';
$wgUploadPath = '/images';

Finally. My images instantly start working and I can upload files again. Thank goodness! This wasn’t a difficult thing, but it is surprising that I couldn’t find any help on the internet for this specific problem. And since I don’t install/perform maintenance on MediaWikis very often, there is a lot of stuff I have to relearn. So here it is, some notes on this issue for my future self, in case I ever need it.

Bed Frame

It recently became apparent that my son was too big for his toddler bed. As we searched for a twin bed at popular stores for children, I found that beds are grossly over priced. The bed my wife liked was over $600, not including the mattress!

We decided that I could make a better quality bed for much less than that, so I did!

Planning

I started out by drawing some hand sketches with rough measurements. I went through the process a few times, each time refining the plan so that it was more complete, and had more accurate dimensions. I also tried to simplify the material selection so there was as little variety in the lumber required as possible.

Bed Plans 2Bed Plans 1

We chose to paint the bed, rather than stain it. This allowed me to choose multiple types of woods which helped cut costs. Here is what was purchased:

Item Usage Quantity Item Price Total Price
4×4, 16ft (Redwood) Bed Posts 2 $13.97  $27.94
1×6, 24 ft Lower Bed Rails 3 $5.91  $17.73
1×4, 8 ft (Spruce) Head/foot board upper rail 1 $1.84  $1.84
1×1, 16ft (Poplar) Mattress Flange 2 $3.50 $7.00
4×8 3/4″ Plywood Mattress support 1 $38.27 $38.27
4×8 1/8″ Plywood Head/foot board material 1 $7.50 $7.50
1″ Slats (molding) Head/foot board decoration 40 Feet $.36 $14.45
Zinsser Primer/Sealer Prime 1 Gallon $19.98 $19.98
Blue Semi Gloss Paint Paint 1 Gallon $26.97 $26.97
Total Cost: $161.68

Total cost after tax: $175.90

Bed Posts

BedPostsThe bedposts are made from 4x4s meant for framing, which come in 8 foot lengths. The posts for the headboard are 4 feet tall, and the posts for the foot board are about 3 feet tall, so there was plenty of wood to get the posts from the 2 lengths that were purchased.

Obviously, the 4×4’s are way too fat for posts on a child’s bed, so I cut them down to 2.7 x 2.7 using a band saw. Next, I sanded them down to 2.5 x 2.5 by taking off .05 inches on each side using a reciprocating drum sander.

The whole bed was designed to be put together using mortise and tenon joints. The next thing I had to do is put in the mortises. For this, I used a mortise and tenon jig that my dad had. I used the largest setting for all of the mortise and tenons I had to do, which made set up pretty easy since I only had to do it once.

With the mortises all cut, I then needed to add a slot for the 1/8″ sheet of wood that will serve as the head and foot boards. I set up the router with the 1/8″ bit, and a depth of .25 inches, and centered it on the mortises, and made made a groove between the upper and lower mortises on all four posts.

Next, I took the hard edge off the corners by running them each through the router, using a small 1/8 inch quarter round bit.

I designed the bed posts to have a decorative groove near the top. The posts will protrude above the upper rail 2.5 inches, so that the protrusion is a perfect, symmetrical cube of material. I used the golden ratio to decide where to put the groove. The groove was made with the table saw, and is just a single kirf width.

Rails

BedRailsThe lower rails for the bed frame are made from the 1×6 planks of wood, and the upper rails for the head and foot boards are made from the 1×4 planks of wood.

The rails will be supported primarily by the mortise and tenons. I set the tenon jig to cut to a depth of 1 inch, so for the final measurement of the wood, one needs to add 2 inches to account for the tenon.

With the rails all cut to length, and complete with tenons, I routed the corners with the same 1/8″ quarter round that was used on the bed posts.

The shorter of the rails which will be used on the head and foot board require a groove for the head and foot board material. I used the same 1/8″ bit that was used on the posts, and added the groove on one side of all 4 rails at a depth of .25 inches.

Finally, some light sanding, and they are ready for paint.

Head and Foot Boards

The head and foot boards are made up of a 1/8 inch sheet of wood that is inset into the bedposts and rails. This will then be covered with decorative slats.

All I had to do was measure the opening, and add 1/2 an inch to account for the groves in which the wood will be set.

Once cut down to the proper sizes, the head board and foot boards can be assembled. This is done by placing one post down on the table with the mortises face up. The upper and lower rails receive some glue on the tenons which are then inserted to the mortises of the posts, and lightly hammered into place.

IMG_3719Glue is applied to the exposed grooves. The headboard material is then slid down from the top until it is seated in the groove on the post. Finally, glue is applied to the other end of the rail’s tenons, as well as the end of the headboard sheet, and the other post is tapped into place. The whole thing is clamped for a few hours until the completed headboard is dry.

IMG_3720I followed this same procedure for the foot board.

IMG_3722Lastly, slats are added. I took the 1 x 1/4 material and cut it down to the proper length to match the height of the inset area on the headboard. I also used extra material to make spacing jigs to help me distribute the slats evenly across the head and foot boards. They were glued in place. In addition to adding visual interest, the slats add a significant amount of stiffness to the otherwise flimsy 1/4″ material.

Painting

IMG_4014The whole bed was dry-fit together and painted all at once. I masked off a strip on the inner side of the long rails where I planned to put in the flanges that will support the mattress board.

IMG_4016I applied one coat of primer using a gravity fed spraying apparatus for the air compressor. This ended up taking forever due to the small amount of paint that the nozzle delivered. It took about an hour, but was reasonably evenly covered.

IMG_4021I let the primer dry completely, and then began applying coats of the blue paint using an airless sprayer. The airless sprayer delivered a much thicker coat, and after 3-4 coats of paint, I had used the whole gallon! It looked pretty good though.

Final Touches

Next, I cut the 1×1″ poplar to length, then glued and screwed it to the interior side of the long rails that had been masked off. This will serve as the flange upon which the mattress board will sit, and carry all the weight.

The very last detail before assembly is to drill the holes in the rails that will hold the headboard and foot board together. I used an angled countersink jig to put 2 shallow-angled holes on either end of both rails that will allow screws to hold it tightly to the posts.

With all the pieces reader to go, the bed frame could be assembled. The rails were lightly hammered into the head board posts, then screwed in place.

The foot board posts are then hammered into place on the rails and screwed in.

The mattress board was then cut to snugly fit between the rails, and then laid on top of the flange with the mattress placed on top.

Just add a pillow and some sheets, and the bed is completely finished!

Results

IMG_0238_e

The bed turned out great. It is super-sturdy. No rocking or creaking. The price was also exceptional. The paint has a few minor blemishes due to my inexperience with painting, but looks pretty good overall.

My son loves his new bed, and it’s sturdy enough for all the antics of a toddler.

© 2007-2015 Michael Caldwell