Accelerating a RISC-V Processor with a Custom Instruction
How to identify a repeated computation, design a custom Instruction Set Extension, integrate it into a pipelined processor, and measure the speedup up to 1.5× acceleration on real RTL.
In the previous post, we built a 5-stage pipelined RV32I processor from scratch in SystemVerilog. We designed it to be modular with clean interfaces, typed control structs, separated pipeline stages, and we ended by asking: where would a custom instruction go? How would the decode, ALU, and control paths change?
This post answers those questions with a concrete example. We take a real workload — a grade scaling and classification loop and implement it first using standard RV32I instructions, identify the repetitive bottleneck, design a custom GRD instruction to eliminate it, modify four files in the processor, and measure the result: a 1.5× speedup from 86 cycles down to 59 cycles. The same methodology applies to any domain-specific acceleration, from signal processing to cryptography to neural network inference.
Part I: The Workload — Grade Scaling with Pass/Fail Classification
The problem is simple. We have an array of five student grades stored as byte values:
| Index | 0 | 1 | 2 | 3 | 4 |
|---|---|---|---|---|---|
| Grade | 28 | 34 | 43 | 50 | 55 |
The processor needs to apply a 25% increase to each grade (computed as grade + grade/4 using integer arithmetic), then classify the scaled result as pass (1) or fail (0) based on a threshold of 60. The expected outputs are:
| Index | 0 | 1 | 2 | 3 | 4 |
|---|---|---|---|---|---|
| Scaled | 35 | 42 | 53 | 62 | 68 |
| Result | 0 | 0 | 0 | 1 | 1 |
Grades 50 and 55 scale above 60 and pass. The rest fail. This is a small workload, but it has the structure that matters for acceleration: a tight loop with a repeated multi-instruction computation applied to every element.
Part II: The Baseline — Standard RV32I Assembly
The first step in acceleration is always to implement the workload using standard instructions and measure the baseline. Here is the complete RISC-V assembly program:
########################################## grades_base = 0x00000000 ##########################################0x0000: addi x10, x0, 0 # x10 = grades base (0x0)########################################## Write grades to memory ##########################################0x0004: addi x5, x0, 280x0008: sb x5, 0(x10)0x000C: addi x5, x0, 340x0010: sb x5, 1(x10)0x0014: addi x5, x0, 430x0018: sb x5, 2(x10)0x001C: addi x5, x0, 500x0020: sb x5, 3(x10)0x0024: addi x5, x0, 550x0028: sb x5, 4(x10)########################################## results_base = 0x00000010 ##########################################0x002C: addi x11, x0, 16 # x11 = results base (0x10)########################################## Setup grading loop ##########################################0x0030: addi x12, x0, 5 # number of grades0x0034: addi x14, x0, 60 # pass threshold########################################## Grade processing loop ##########################################0x0038: grade_loop: lbu x5, 0(x10) # load grade0x003C: srli x6, x5, 2 # grade >> 20x0040: add x6, x5, x6 # new_grade = g + (g >> 2)0x0044: slt x7, x6, x14 # new_grade < 60 ?0x0048: bne x7, x0, fail # branch to fail########################################## Pass path ##########################################0x004C: pass: addi x8, x0, 10x0050: sb x8, 0(x11)0x0054: jal x0, next########################################## Fail path ##########################################0x0058: fail: sb x0, 0(x11)########################################## Loop update ##########################################0x005C: next: addi x10, x10, 1 # next grade0x0060: addi x11, x11, 1 # next result0x0064: addi x12, x12, -10x0068: bne x12, x0, grade_loop########################################## End of program (halt) ##########################################0x006C: addi x0, x0, 0 # nop (halt)
What happens in the hot loop
Look at the code between grade_loop and next. For each grade, the processor executes:
lbu— load the grade byte from memorysrli— shift right by 2 (integer divide by 4)add— compute scaled grade = grade + grade/4slt— compare against threshold 60bne— branch to fail or fall through to passaddi+sb— store the result (1 or 0)- Three
addi+bne— update pointers and loop counter
The scaling and classification in steps 2 through 5 takes four instructions per element. On a pipelined processor with forwarding and stalls, this translates to multiple cycles per iteration consumed by a computation that is fundamentally the same every time: take a value, scale it, compare it, produce a 0 or 1.
Baseline measurement
Running this program on the processor in Vivado simulation, the total execution time is 86 clock cycles (measured from the first instruction after reset to the final store).
Part III: Identifying the Optimisation Opportunity
The acceleration methodology is straightforward: analyse the code, identify repetition, and move critical logic into hardware.
In this program, applying the scaling factor and checking whether the result passes the threshold is a repetitive operation composed of several instructions which are srli, add, slt, bne. These four instructions appear in the hot path of every loop iteration. If we had a single dedicated instruction that receives a grade and directly returns pass (1) or fail (0), it would eliminate the need for all of those internal loop operations.
That instruction is GRD:
grd rd, rs1
The GRD instruction takes a grade value from source register rs1, internally computes (rs1 + (rs1 >> 2)) >= 60, and writes the result 1 if pass, 0 if fail to the destination register rd. One instruction replaces four.
Part IV: Designing the Custom Instruction Encoding
RISC-V reserves opcode space specifically for custom extensions. However, the GRD instruction fits naturally into the existing R-type encoding used by all register-register ALU operations. The R-type format is:
| funct7 (7) | rs2 (5) | rs1 (5) | funct3 (3) | rd (5) | opcode (7) || 31 25 | 24 20 | 19 15 | 14 12 | 11 7 | 6 0 |
The existing R-type instructions distinguish themselves using the funct7 and funct3 fields. For example, ADD is {funct7[5], funct3} = 4'b0000 = 4'h0, while SUB is 4'b1000 = 4'h8. The full mapping:
| Instruction | funct7 | funct3 | {funct7[5], funct3} |
|---|---|---|---|
| ADD | 0000000 | 000 | 4’h0 |
| SUB | 0100000 | 000 | 4’h8 |
| SLL | 0000000 | 001 | 4’h1 |
| SLT | 0000000 | 010 | 4’h2 |
| SLTU | 0000000 | 011 | 4’h3 |
| XOR | 0000000 | 100 | 4’h4 |
| SRL | 0000000 | 101 | 4’h5 |
| SRA | 0100000 | 101 | 4’hD |
| OR | 0000000 | 110 | 4’h6 |
| AND | 0000000 | 111 | 4’h7 |
| GRD | 0100000 | 001 | 4’h9 |
The GRD instruction uses funct7 = 0100000 and funct3 = 001, giving a combined selector of 4'h9. This is the same pattern as SRA vs SRL, the funct7[5] bit distinguishes GRD from SLL, which shares the same funct3 = 001. The rs2 field is set to 5'b00000 since GRD only uses one source operand.
Part V: Modifying the Processor
The modular design from the first post pays off here. Adding the GRD instruction requires changes to exactly four files, and each change is small and localised.
1. risc_pkg.sv — Add the ALU enum variant
typedef enum logic [3:0] { ALU_ADD, ALU_SUB, ALU_SLL, ALU_SLT, ALU_SLTU, ALU_XOR, ALU_SRL, ALU_SRA, ALU_OR, ALU_AND, ALU_COPY_B, ALU_NOP, ALU_GRD // <-- new} alu_op_t;
One line. The packed struct types, pipeline registers, forwarding unit, hazard unit, and writeback mux do not change at all — they already carry alu_op_t through the pipeline generically.
2. alu_control.sv — Map the encoding to the new ALU op
The ALU control decoder already switches on {funct7[5], funct3} for R-type instructions. The only change is adding the GRD case:
OPCODE_OP: begin unique case (funct3) F3_ADD_SUB: alu_op = funct7[5] ? ALU_SUB : ALU_ADD; F3_SLL: alu_op = funct7[5] ? ALU_GRD : ALU_SLL; // <-- new F3_SLT: alu_op = ALU_SLT; F3_SLTU: alu_op = ALU_SLTU; F3_XOR: alu_op = ALU_XOR; F3_SRL_SRA: alu_op = funct7[5] ? ALU_SRA : ALU_SRL; F3_OR: alu_op = ALU_OR; F3_AND: alu_op = ALU_AND; default: alu_op = ALU_NOP; endcaseend
When the opcode is R-type and funct3 = 001 and funct7[5] = 1, the ALU control selects ALU_GRD instead of ALU_SLL. One line changed.
3. alu.sv — Implement the GRD operation
The ALU gets one new case in its combinational unique case block:
ALU_GRD: result = ((lhs + (lhs >> 2)) >= 32'd60) ? 32'd1 : 32'd0;
This single line performs the entire grade scaling and classification. It takes lhs (the grade from RS1), computes lhs + (lhs >> 2) (the 25% increase using integer arithmetic), compares against 60, and produces a 1 or 0. The hardware cost is one adder, one shifter, one comparator, and one mux — all of which the ALU already contains for other operations. The incremental area cost is minimal.
4. control_unit.sv — No changes needed
This is the important part. The control unit already handles all R-type instructions identically: set op_a_sel = OP_A_RS1, op_b_sel = OP_B_RS2, wb_sel = WB_ALU, rf_wr_en = 1, uses_rs1 = 1, uses_rs2 = 1. The GRD instruction is R-type, so it flows through the same control path. Forwarding, hazard detection, and writeback all work without modification because they operate on the generic alu_op_t type, not on specific instruction identities.
This is exactly what the modular architecture was designed for.
Part VI: The Accelerated Program
With the GRD instruction available, the inner loop becomes dramatically simpler:
########################################## Grade processing loop using ISE ##########################################0x0034: grade_loop: lbu x5, 0(x10) # load grade0x0038: grd x6, x5 # ISE: x6 = 1 if pass, 0 if fail0x003C: sb x6, 0(x11) # store result0x0040: addi x10, x10, 1 # grades++0x0044: addi x11, x11, 1 # results++0x0048: addi x12, x12, -1 # count--0x004C: bne x12, x0, grade_loop########################################## End (halt) ##########################################0x0050: addi x0, x0, 0 # nop
The four-instruction sequence (srli → add → slt → bne) has been replaced by a single grd x6, x5. The branch-based pass/fail decision is gone — the result is now directly computed by the ALU. The entire loop body, including memory access and pointer updates, is now seven instructions instead of eleven.
Machine code encoding
The grd x6, x5 instruction encodes as:
funct7 = 0100000rs2 = 00000rs1 = 00101 (x5)funct3 = 001rd = 00110 (x6)opcode = 011001132'b0100000_00000_00101_001_00110_0110011= 32'h40029333
In little-endian byte order for the .mem file: 33 93 02 40.
Part VII: Simulation and Results
Both baseline and accelerated programs were simulated in Vivado 2025.2 using self-checking testbenches that count clock cycles and verify the output memory contents.
Correctness verification
The grd_test_tb testbench verifies all five result bytes:
---- GRD result check ----mem_byte[16] = 0 (expected 0) PASSmem_byte[17] = 0 (expected 0) PASSmem_byte[18] = 0 (expected 0) PASSmem_byte[19] = 1 (expected 1) PASSmem_byte[20] = 1 (expected 1) PASSGRD TEST: PASS
The custom instruction produces identical results to the multi-instruction baseline.
Cycle count comparison
| Version | Execution cycles | Loop body (per iteration) |
|---|---|---|
| Baseline (standard RV32I) | 86 cycles | ~4 ALU + branch + stores |
| Accelerated (with GRD ISE) | 59 cycles | 1 ALU + stores |
| Speedup | 1.5× | — |
The GRD instruction eliminates 19 cycles across 5 iterations — roughly 3–4 cycles saved per iteration, which corresponds exactly to removing the srli, add, slt, and bne instructions from the hot path. On a larger dataset, the per-iteration savings compound: a 1000-element array would save approximately 3,000–4,000 cycles.
The Vivado waveform confirms the execution: completion_cycle = 59 for the GRD version versus cycle_count = 90 for the baseline (both including reset overhead), with the done signal asserting cleanly after the final store and pass_count = 5 matching all expected values.
Part VIII: The Acceleration Methodology
This example demonstrates a general methodology that applies far beyond grade scaling:
Step 1 — Profile the workload. Identify the hot loop and count the instructions per iteration. In this case, 4 instructions were consumed by scaling and comparison logic on every iteration.
Step 2 — Identify the repeated pattern. The scaling and classification is a fixed, deterministic computation. The same arithmetic applied to different data. This is the hallmark of a good ISE candidate: repetitive, compute-bound, and expressible as a single operation.
Step 3 — Design the instruction encoding. Choose an encoding that fits the RISC-V framework. R-type is natural for register-to-register operations. Use the reserved funct7/funct3 space to avoid conflicts with standard instructions.
Step 4 — Modify the minimal set of RTL files. In a well-structured processor, this should be localised — package, ALU control, and ALU for simple single-cycle operations. The pipeline infrastructure (forwarding, hazards, writeback) should not need changes.
Step 5 — Verify functional equivalence. The accelerated program must produce identical results to the baseline. Self-checking testbenches with explicit expected-value comparisons catch regressions immediately.
Step 6 — Measure the speedup. Count cycles, not instructions. Pipeline effects (stalls, flushes, forwarding latency) mean that instruction count alone does not predict cycle count.
Part IX: Implications for Edge AI
The GRD instruction is a toy example, but the pattern it demonstrates is the same pattern used in production AI accelerators:
MAC (multiply-accumulate) instructions collapse a multiply and an add into a single cycle exactly analogous to GRD collapsing shift, add, compare, and branch.
Dot-product instructions process entire vectors in a single operation, replacing loops of individual multiplies and accumulates.
Quantised arithmetic instructions — INT8 multiply, saturating add, fused scale-and-clip follows the same principle: identify a repeated multi-instruction pattern in the inference hot path and replace it with dedicated hardware.
The processor we built is ready for this. Adding a MAC rd, rs1, rs2 instruction that computes rd = rd + (rs1 * rs2) would follow the same four-file modification pattern. A RELU rd, rs1 instruction that computes max(0, rs1) would be even simpler, also is adding a single comparator and mux in the ALU. The modular architecture means each extension is incremental, testable, and does not disturb the existing instruction set.
What Comes Next
This post demonstrated single-cycle custom operations that complete in one pipeline stage. Future work will explore multi-cycle custom operations (where the hazard unit must stall upstream stages), tightly coupled accelerator blocks (dedicated functional units alongside the ALU), and memory-side acceleration (custom load/store variants for structured data access patterns).
The complete SystemVerilog source including the modified risc_pkg.sv, alu.sv, alu_control.sv, baseline and accelerated .mem files, and both testbenches with cycle-counting instrumentation is available on GitHub/risc_v.
Published on neuralonedge.com