SonarQube migration issue- Jenkins Using old URL

SonarQube migration issue- Jenkins Using old URL

I recently migrated a SonarQube server from one server to another in order to scale out the service to our dev team. All went well until builds failed due to them looking at both the old and new server URLs for the Sonar results and so I’m writing some notes here to help me (and others) out in the future if I hit this again.

I installed the same version of SonarQube on the new application server that is on the old server. The database was not being moved, just the application server (the server with the Sonar service running).

After installation I ensured that the same custom config settings were made onthe new server as had been made on the old server, and ensured that the same plugins were in place. I then stopped the Sonar service on the old server and started the service on the old box.

Once Sonar was confirmed to be up and running and connecting to the Database and showing the project dashboards I updated the Jenkins server configuration to point to the new box. All looked good so I ran a build, and then got this (log truncated)….

 
INFO: ANALYSIS SUCCESSFUL, you can browse http://OLDSERVER/dashboard?id=123
INFO: EXECUTION SUCCESS
The SonarQube Scanner has finished
Analysis results: http://NEWSERVER/dashboard/index/APP
Post-processing succeeded.
withSonarQubeEnv
waitForQualityGate
java.lang.IllegalStateException:
Fail to request http://OLDSERVER/api/ce/task?id=3423
at org.sonarqube.ws.client.HttpConnector.doCall(HttpConnector.java:212)

Bizarely the Jenkins build has managed to use both the new Sonar URL and the old one.The upload was successfull to the new server but some of the links for the resport point to the old server. Also the Quality Gate check whcih validates that the Sonar Quality Gate was successfull has tried to read the report on the OLD server and therefore failed as its not there (because its on the new sonar URL).

After checking Jenkins for any reference to the old Sonar server and restarting the service to clear any caches I was still getting the error. Eventually I ran a Jenkins build and interactively peeked into the Jenkins workspace on the Jenkins Slave and in there is an auto generated file containing lots of Sonar config settings. This file, SonarQubeAnalysisConfig.xml, is created during the Jenkins build initialisation stage. In the file I found references to the new Sonar URL but also this property pointing to the old URL:

  sonar.core.serverBaseURL  

This value is set in SonarQube configuration and is not dynamic and so will not be updated when you migrate the server or change the server URL/port. To change it open SonarQube > Administration > Configuration > General and change Server base URL to your new URL (e.g. http://yourhost.yourdomain/sonar). It says that this value is used to create links in emails etc but it in reality is also used to integate results.

Advertisements

SonarQube: Unit Test Results Not Shown

SonarQube: Unit Test Results Not Shown

Recently whilst building Jenkins CI pipeline, with SonarQube static analysis, the JUnit unit test results were not being included in the Sonar dashboard results. The Jacoco based test coverage results were being included fine but not the actual test pass/fail percentage.

sonardash2

After digging into the log for the Jenkins build I found this warning being logged for the SurefireSensor (the Sonar sensor responsible for scanning JUnit XML reports for results):

[sonar] 10:26:34.534 INFO - Sensor SurefireSensor
[sonar] 10:26:34.534 INFO - parsing /apps/jenkins2/var/lib/jenkins/workspace/abc/code_master/examplecode/UnitTest/junit
[sonar] 10:26:34.864 DEBUG - Class not found in resource cache : com.rh.examplecode.UIMapperTest
[sonar] 10:26:34.864 WARN - Resource not found: com.rh.examplecode.UIMapperTest

The JUnit XML reports were being found and parsed fine but when it’s looking for the actual test code (the *.java code) it could not be found by the scanner and hence it throws the warning. It turns out that the java code for the tests is required in order analyse the JUnit results files and so you need to tell Sonar where to find the source code for the tests. How? Well this is done via the sonar.tests” property which is a comma-separated list of filepaths for directories containing the test code (the *.java files not *.class files). For example:

sonar.tests = "/UnitTests/junit"

Set this property alongside the other parameters for Sonar, for example:

sonar.projectBaseDir="${WORKSPACE}/exampleApp"
sonar.projectKey="testbuild1"
sonar.projectName="testBuild"
sonar.sourceEncoding="UTF-8"
sonar.sources="src/main/java/com/rh/examplecode/"
sonar.junit.reportsPath="ReportsXML/"
sonar.tests= "/UnitTests/junit"
sonar.jacoco.reportPath="target/jacoco.exec"
sonar.jacoco.reportMissing.force.zero="true"
sonar.binaries="build/com/rh/"

After this change the Sonar scanner will run and this time find the test source code, enabling it to complete the analysis. The log should report something like this:

[sonar] 13:10:20.848 INFO - Sensor SurefireSensor
[sonar] 13:10:20.848 INFO - parsing /apps/jenkins2/var/lib/jenkins/workspace/abc/code_master/ReportsXML
[sonar] 13:10:21.022 INFO - Sensor SurefireSensor (done) | time=10ms

And you should now have your unit test success/failure results in your unit test widgets in the projects Sonar dashboard, like so:

sonardash1

Some SonarQube Upgrade Issues & Fixes

Some SonarQube Upgrade Issues & Fixes

I recesq-ci-72xntly upgraded a SonarQube server installation from v5.6.2 to v6, and unfortunately hit a few issues along the way which I thought I’d share here in case others experiences the same issues. All were resolved in the end and if you are yet to be running SonarQube to analyse your software assets please don’t be put off my these small issues. SonarQube is an outstanding tool to have in your Quality Control armoury and it is really incredibly easy to set and run. In fact you can download it and run it  straightaway in under two minutes without installing anything (check out this link Get Started in Two Minutes to learn how).

Anyway the first problem I hit with the upgrade was an error message in the log when the service was trying to connect to the database (in my instance an MS SQL Server):

Unsupported JDBC driver provider: jtds 

Apparently support for jtds was changed to a bundled version by  SonarQube at some point and so it needs to be removed from the connection string:

Original connection string:
sonar.jdbc.url=jdbc:jtds:sqlserver://ServerName;instance=sonar;databaseName=sonar

New connection string:
sonar.jdbc.url=jdbc:sqlserver://ServerName;instance=sonar;databaseName=sonar

This change made I still could not connect to the database but this time due to different error, which was:

Can not connect to database. Please check connectivity and settings. The TCP/IP connection to the host ServerName1, port 1433 has failed. Error: “Connection refused: connect. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.”

After verifying the database was indeed up, running, not blocked by a firewall and indeedsqlserverconfigmgrexample open on the specified port I found that I had to turn off dynamic ports on my Sonar DB server. To do this open the SQL Server Configuration Manager application, under SQL Server Network Configuration – Protocols for Sonar, right click TCIP/IP and choose properties. Under IP Addresses ensure that TCP Port is 1433 for all entries (including IPAll) AND ensure that TCP Dynamic Ports is blank. My TCP Dynamic Ports value was “0” which actually enables dynamic ports! After this change DB connectivity was successful.

At this point the auto-upgrade step failed and after integrating the logs I found this problem:

Cannot resolve the collation conflict between “Latin1_General_CI_AS” and “Latin1_General_CS_AS” in the equal to operation.

After some googling I hit this very useful Stack Overflow post where the problem is explained. I choose to manually update the database collation (option 3). After running the suggested query I was able to work out the indexes that needed to be dropped and recreated to enable the collation to be updated.

After this I deleted the data out of the SonarQube temp folder (ensuring that the Sonar Service had been stopped) and restarted the service and this triggered the upgrade process again which this time completed successfully.