2 posts tagged with "on the job"

Context​

In software engineering, we often encounter work that is repetitive in nature. Examples of such tasks include setting up a new project, conducting a series of manual tests, or making a new release. While some tasks can be automated, reducing them to a mere click of a button or execution of a script, others are more complex and demand careful attention during their execution. Given that these tasks may not always be performed by the same individual, it's crucial to determine how to ensure correct execution every time.

Problem​

Here are several issues that can arise when tasks are poorly managed:

• The task may not be executed correctly.
• Completion of the task may take longer than anticipated.
• The task may become frustrating to execute.
• The task may not be executed at all.

Solution(?)​

A straightforward solution to this problem is to maintain well-documented tasks. Such documentation should detail the task comprehensively, including steps for execution, the anticipated outcome, and, if applicable, some troubleshooting guidance. To ensure the documentation remains relevant, it is essential to:

• Ensure the documentation is easily accessible (i.e., not buried and subsequently forgotten).
• Make the documentation simple to update (ideally without requiring approval).

Documenting tasks in the location where they are defined and assigned appears to be a viable strategy. The issue/ticket description often serves as the first and possibly the last reference point when someone is assigned a task. It is readily updateable if, while following the instructions, someone identifies an error or a more efficient method. A critical feature of this approach is the ability to clone the issue/ticket, facilitating the future repetition of the task.

However, this method has its limitations:

• When a task is too complex to be fully described within a single issue/ticket, it can render the issue/ticket less effective due to a level of indirection to the actual documentation. Similarly, if a task evolves (e.g., changes in expectations or implementation), the issue/ticket may become outdated as discussions shift to the place of change (codebase) or instant messaging platforms.
• Maintaining synchronized information across tasks can be challenging, especially if a task shares a prerequisite setup with several others. It becomes difficult to update all relevant tasks en masse upon recognizing a necessary change in setup.
• A task may need to be necessarily brief if it is part of a larger project, thus lacking comprehensive information. In such cases, separate documentation may be required to provide a high-level overview of all tasks.
• Typically, task systems do not offer the full feature set of a documentation system, such as the ability to include inline comments or facilitate real-time collaboration.

So, what's the best way to keep task descriptions up to date...?

Context​

Balancing the ease of implementation with the correctness of a solution is a complex trade-off. When developing a package to be used in a CI/CD pipeline for multiple repositories, I encountered the challenge of deciding how to handle the package's versioning strategy within the CI process.

Problem​

Pinning the package to a specific version in the Jenkins script for each repository ensures that the CI process is stable and predictable. However, this approach necessitates manual intervention for each repository whenever a new package version is released, which can be problematic, especially considering the following pragmatic factors:

• Diverse repositories managed by different teams, where gaining approvals for changes can be time-consuming and laborious.
• The package is still under active development, with new versions released frequently.

On the other hand, always using the latest package version in CI pipelines simplifies updates but risks unexpected disruptions. This approach can eliminate the need for manual updates to many repositories but also introduces the risk of breaking changes, leading to failing CI pipelines across various repositories, which can have adverse consequences:

• Unexpected disruptions for developers in their branches or PRs.
• Resistance from developers, possibly leading to the removal or ignoring of this CI step.

Discussion​

Finding a balance between the two approaches is crucial, and importantly, it requires a deeper understanding of the underlying problems and whether we can address them in a more fundamental way. Here are some hidden issues behind this problem:

• Why do cross-team, multi-repo changes intimidate and slow down processes?
• Are there ways to automate the creation of similar changes to multiple repositories?

For the first problem, it might be a management issue where a standard procedure can be devised to guide the process of assigning responsibilities and gaining approvals for cross-repo changes within the organization/team. For the second problem, it might require additional tooling to address the repetitive nature of the changes. It could also suggest that this configuration might benefit from more centralized control, where a single repository can manage the package version for all connected repositories.

Retrofitting​

Before diving into what I would consider a better approach, I would like to discuss how we can retrofit the easy solution (always install and use the latest package version in CI pipelines):

• Commit to backward compatibility: Avoid breaking changes at all costs.
• Support previous x versions:
  • Maintain backward compatibility for the previous x versions.
  • Notify users of required upgrades without breaking their current setup for a reasonable period.
• Provide upgrade support: Assist repositories in adapting before releasing breaking changes and updating the package version after new releases.

Solution​

The solution I propose is to pin a specific package version in CI and upgrade only when necessary. To address the issues, I would also propose the improvement items mentioned in the discussion section:

• To deal with the troublesome manual updates:
  • Create codemod-like tools or scripts to automate the process.
  • Revert the usage model to more centralized control, where a single repository can configure the package version and the repositories that will use this package in the CI pipeline.
• To deal with cross-team, multi-repo changes:
  • Find out the established process for proposing and getting support for cross-repo changes, which may involve sharing the proposal in a forum/meeting, getting the owners' support, and then proceeding with the changes with known assigned liaisons for each repository.