Managing Calculations on Your Desktop: Leveraging the Lightweight Job Scheduler FBQueue

Running too many calculations simultaneously can cause your PC to freeze, or you might waste time by forgetting to start the next job after one finishes overnight. These are common issues in research and data analysis.

While job schedulers like Slurm or PBS, commonly used on supercomputers, solve these problems, deploying them on personal desktops or small shared servers has been difficult due to setup complexity and the requirement for root access.

In this post, we introduce FBQueue, a lightweight job scheduler that can be easily deployed in such environments.

Why You Need a Job Scheduler for Your Desktop

Manually running multiple scripts or relying solely on background execution (e.g., using &) presents several challenges:

1. Resource Contention: Running more tasks than your PC’s core count can severely degrade system responsiveness, hindering other work.
2. Idle Gaps: If a calculation finishes late at night or while you are away, resources remain idle until you manually start the next task.
3. Dependency Control: Manually managing workflows, such as “run Task B only after Task A finishes successfully,” is tedious and prone to error.

By introducing a job scheduler, you can automatically run submitted tasks in sequence within defined resource limits (such as CPU core count).

FBQueue Features: Easy Setup and Clean Operation

FBQueue is specifically designed for personal use and small groups:

• No Root Access Required: It runs entirely within your home directory; just place the binary and start.
• Easy to Set Up: No database configuration or complex installation is needed—it’s a standalone binary.
• Clean Uninstallation: To uninstall, simply delete the executable and the ~/.fbqueue directory where settings and logs are stored.

Practical Example: Scheduling Computation Tasks

If you have multiple heavy analysis scripts (analysis.sh) that take hours to run, you can manage them with FBQueue as follows:

1. Submitting Jobs

# Submit as tasks consuming 8 CPU cores each
fbqueue sub --cost 8 ./analysis_1.sh
fbqueue sub --cost 8 ./analysis_2.sh
fbqueue sub --cost 8 ./analysis_3.sh

2. Monitoring Status

After submitting jobs, you can check their current execution status or pending tasks at any time with the following command:

fbqueue stat

For instance, if your PC’s capacity is set to “16,” the first two tasks will start running (RUN) immediately, while the third will automatically wait (PEND). When a preceding task finishes, the waiting task starts automatically. This allows you to keep calculations running without overloading the system or leaving resources idle.

Usage on Shared Servers and Arm Environments

FBQueue is also useful on shared servers that lack a formal scheduler. By setting your allowed resource usage (e.g., CPU core count) as your capacity, you can efficiently run background tasks without interfering with other users.

Furthermore, version 0.9.3 supports Linux Arm64 (AArch64), enabling the same workflow on Raspberry Pi, AWS Graviton instances, or Jetson devices.

Getting Started

You can download FBQueue from the GitHub releases page. Simply add the executable to your system PATH, and it’s ready to use.

FBQueue Details & Download