Troubleshooting Performance and Stability Issues
In cases when JIRA's performance deteriorates or if the system becomes unstable or unresponsive, it is important to achieve two goals:
Bring system back to normal in the shortest amount of time.
Collect information that would help analyze the problem and make sure it does not appear again.
The second goal is strategically very important, however, it might get overlooked in a rush to make things work "now". For example, JIRA administrator may be inclined to restart a stuck JIRA instance quickly in order for it to get back to working state as fast as possible. But if thread dumps are not collected, the developers will never know where JIRA was stuck, so the same problem may happen again.
The first goal is of course also very important. Sometimes JIRA administrator manages to restore system functioning, sometimes help from Atlassian and Tempo support teams is needed. Support engineers and developers would typically take into account all information they have, analyze it and try to pinpoint the source of the problem. Often additional information is required from the JIRA administrator, and sending requests and replies back and forth takes precious time.
This article is intended to provide JIRA administrators with advice about how to collect maximum information about a performance or stability problem, when that problem happens. The list is not intended to be complete, additional information may still be needed, however, providing all listed information gives a good chance that a support engineer will be able to identify a problem and provide advice sooner.
Thread Dumps
Thread dumps are the most important information when system is unresponsive or has performance issues. They allow to peek into what's going on inside JIRA's JVM process.
Please refer to Atlassian documentation on generating a thread dump for instructions of manually capturing a thread dump on the server.
Thread dumps are also a part of the Support Zip (3 dumps are generated in one zip), however, generating a support zip might be unavailable if JIRA is hanging.
For best diagnosis, please collect 5-6 thread dumps with 3-4 second interval.
Please collect 5-6 thread dumps with 3-4 second interval.
Verbose Logging
If the problem has temporary but reproducible manner, you can turn on verbose logging so that the engineers can gather more information from the logs. To do so:
Open Administration | System | Logging and Profiling page.
Enter STRUCTURE TROUBLESHOOTING into the Optional Message field, turn on Log Rollover and press Mark.
Scroll down and click Configure logging level for another package, enter package name com.almworks then select logging level DEBUG and click Add.
Then you can try to reproduce the problem and collect the support zip.
Do not forget to turn off the DEBUG logging after the problem has been resolved, otherwise you may get too many messages in the logs during normal operation.
Support Zip
Support zip is the most important thing after thread dumps. It allows engineers to have full understanding of the environment and retrospect using the logs into what was going on. If you had Verbose Logging on before problem appeared, it gives even more details.
To collect a support zip:
Open Administration | System | Atlassian Support Tools, switch to Support Zip tab.
Open Administration | System | Support Tools, switch to Create Support Zip tab. Select all options. Click Create.
Download the resulting ZIP file and send it to the support teams: either attach it to the ticket, or, if the file is large, request a URL for uploading.
On JIRA Data Center, collect Support Zips on each node.
Browser Console Log
If the problem seems to be on the client side, in the browser – if there are errors or if the browser is hanging or some button or link does not respond, check out the browser's error console. Depending on the browser type, the console may be opened with different menus or keyboard shortcuts.
Reproduce the problem
Copy all contents from the console and send it to support.
Also, please include browser type and version, as well as the information about operating system.
HAR Report
HAR report is also taken on the browser and contains logs of network communications with the server. Use this log to provide information that can help troubleshoot issues with slow loading of data or general slow responsiveness on the client side.
Use Google Chrome
Open menu, More Tools | Developer Tools.
Switch to Network tab
Reproduce the problem
Right click in the table and select "Save as HAR with content..." or "Copy All as HAR".
Paste or save HAR as a file.
HAR with content provides more information but it may contain your JIRA's data. Review the contents before sending it out to support.
Screenshots or Video
When there's a visible and informative behavior demonstrated by the system, a screenshot or a video showing the problem would go a long way in getting the support engineers understand the issue.
You can use operating system's native tools to capture a video, or install a third-party tool for that. Feel free to ask Tempo Support for recommendations if you don't have preferable screen capturing tool.
See also: Monitoring and Troubleshooting Structure Usage