Non-Programmers Taking Part in Peer Reviews and Inspections

The concept of the pull request and peer review has become almost universally known among software engineering teams. For those unfamiliar, the pull request is where a developer submits their work for integration into the master/main code base. The peer review involves getting a second person to look at this work before it is integrated. While organizations will layer additional processes, tools, and automated checks on top of this process, the basics are nevertheless at the centre of many organizations' software development work. Unfortunately, the tools often used for the pull request and peer review are not broadly applicable to other parts of the software development process (outside of the source code). For example, GitHub is used by many organizations to manage the review process. This tool is focused on code changes, and ill-suited for other work products like user stories, requirements, test plans, higher-level design documentation, reports from business analysts, etc. Collaborator, by SmartBear Software, facilitates a transparent, consistent, and cross-disciplinary approach to software development work product reviews. While this is the core value proposition, the broader vision for Collaborator, as the name suggests, is to encourage collaboration among internal stakeholder groups responsible for delivering software [2].

Exploring the Tool

While some of the SmartBear products are available for on-premises implementation, Collaborator is only available as a SaaS offering [2]. A trial can be started from Many companies in the industry provide sample data for software trials/evaluations, but this is not the case for Collaborator. I deliberately avoided using the trial getting started guide, in order to get a better grasp of the usability and friendliness of the software. As they say, a user interface is like a joke - if you need to explain it, it’s not good.

Trial-Collaborator-Instance Figure 1: Trial Collaborator Instance

Approach to Software Inspections

The inspection approach can be better understood through an example. If you are starting from the blank trial system, the review process proceeds as follows:

  1. Click “New Review”
  2. Fill out the new review form
    • Minimum configuration is setting an Author and a Reviewer
    • Add a test file to the Review Materials section of a SCM isn’t configured
  3. Utilize the platform for discussion and collaboration
    • Discussion is possible on the review level, file level, and LOC level
    • Transition the ticket through to annotation and inspection stages when discussion is complete
  4. Provide an approval using the checkbox icon in the “Chat” section
  5. Push the “Move to Completed” button to close the review

Some analysis features, such as code/file deltas are available. Additionally, the reviews can be standardized and integrated with other software system using features like checklists, SCM systems, and custom integrations (e.g. JIRA). There are no obvious limitations to the types of work products which can be reviewed. Of course, all physical work products must be digital (or somehow digitized) for a review. Additionally, some of the analysis features like deltas analysis will not work for images, PDFs, and a variety of other work product formats.


Figure 2: New Review Form


Figure 3: Inspection Stage of Review


Figure 4: Approving the Work Products in the Review


Figure 5: Reviews List

Automation Capabilities

The integrations list is substantial, but not overwhelming. Integrations include Git, Visual Studio, JIRA, Team Foundation Server, Subversion, Perforce, GitHub, AccuRev, Rational Team Concert, ClearCase, FogBugs, BugZilla, Eclipse, Microsoft Office, HipChat, Bitbucket, GitLab, and PTC Integrity [1]. Depending on the integrations installed, Collaborator automations may include:

  • Automated review creation triggered by version control systems
  • CRUD operations on reviews within an external tool (e.g. Visual Studio IDE)
  • Pushing and pulling data from ticketing systems
  • Connection with external file storage systems
  • Defect handling in bug trackers
  • Handle document reviews in a more intuitive way with the Microsoft Office integrations
  • Automated notifications

Final Thoughts

SmartBear’s Collaborator is just one tool of many that facilitate collaboration, review, and feedback among software development teams. The interface feels a bit dated and unpleasant to use, but the multitude of integrations and relative simplicity of the tool may make it valuable for some software teams. More broadly and importantly, the concept of including additional stakeholders in work product reviews can accelerate learning and improve team agility. A collaborative, cross-functional team is a team which can quickly adapt to customer needs.


[1] Supported Integrations, Tools and Controls. (n.d.). Retrieved from

[2] Peer Code Review You Can Trust. (n.d.). Retrieved from