regocpp-builtins

rego-cpp Built-in Function Development

Add, update, and remove OPA Rego built-in functions in rego-cpp.

When to Use

• Implementing a new builtin (replacing a placeholder or adding from scratch)
• Adding a new OPA builtin namespace (new src/builtins/<namespace>.cc file)
• Updating builtin declarations to track a new OPA version
• Replacing placeholder stubs with real implementations
• Removing deprecated builtins
• Debugging builtin dispatch or registration issues

Architecture Overview

Built-in functions follow a three-layer architecture:

BuiltInsDef::lookup(name)          ← Dispatch layer (src/builtins.cc)
  → builtins::<namespace>(name)    ← Namespace router (src/builtins/<ns>.cc)
    → <function>_factory()         ← Factory (returns BuiltIn with decl + behavior)
      → <function>(args)           ← Implementation (unwrap args, compute, return)

Key Files

The Binary Dispatch Tree

BuiltInsDef::lookup in src/builtins.cc uses a hand-coded binary search tree (generated by src/builtins/binary_tree.py) that routes the namespace prefix of a builtin name to the corresponding namespace function. When adding a new namespace, this tree must be regenerated or manually updated.

Special case: "io" prefix routes to builtins::jwt(name) for io.jwt.* builtins.

Procedure

Adding a New Builtin to an Existing Namespace

1. Read the existing namespace file (src/builtins/<namespace>.cc) to understand the patterns in use.

2. Write the implementation function:

   Node my_func(const Nodes& args)
   {
     // Unwrap and validate arguments
     Node x = unwrap_arg(args, UnwrapOpt(0).type(JSONString).func("namespace.my_func"));
     if (x->type() == Error)
       return x;
   
     // Extract values
     std::string val = get_string(x);
   
     // Compute result
     std::string result = do_something(val);
   
     // Return wrapped result
     return JSONString ^ result;
   }
   

3. Write the factory function:

   BuiltIn my_func_factory()
   {
     const Node my_func_decl = bi::Decl
       << (bi::ArgSeq
           << (bi::Arg << (bi::Name ^ "x")
                       << (bi::Description ^ "input string")
                       << (bi::Type << bi::String)))
       << (bi::Result << (bi::Name ^ "y")
                      << (bi::Description ^ "result description")
                      << (bi::Type << bi::String));
     return BuiltInDef::create({"namespace.my_func"}, my_func_decl, my_func);
   }
   

4. Register in the namespace router (the public function at the bottom of the file):

   BuiltIn namespace_func(const Location& name)
   {
     // ... existing dispatches ...
     if (view == "my_func")
     {
       return my_func_factory();
     }
     return nullptr;
   }
   

5. Run OPA conformance tests:

   cd build && ./tests/rego_test -wf opa/v1/test/cases/testdata/v1/<testdir>
   

Adding a New Namespace

When adding an entirely new OPA namespace (new .cc file):

1. Create src/builtins/<namespace>.cc following the pattern of existing files. Include the anonymous namespace for internal functions and the rego::builtins namespace for the public dispatch function.

2. Declare the dispatch function in src/builtins/builtins.hh:

   namespace rego::builtins {
     BuiltIn my_namespace(const Location& name);
   }
   

3. Add to the dispatch tree in src/builtins.cc — find the correct position in BuiltInsDef::lookup based on the namespace prefix string and add the routing branch. Alternatively, regenerate the tree using src/builtins/binary_tree.py.

4. Add the source file to src/CMakeLists.txt:

   set( SOURCES
     # ... existing sources ...
     builtins/<namespace>.cc
   )
   

5. Rebuild and test.

Replacing a Placeholder with a Real Implementation

Many builtins are registered as BuiltInDef::placeholder(...) which returns an error message when called. To replace:

1. Keep the existing declaration (bi::Decl << ...) — it defines the argument and return types.

2. Write the implementation function that takes const Nodes& args and returns a Node.

3. Change the factory from:

   return BuiltInDef::placeholder({"name"}, decl, "message");
   

   to:

   return BuiltInDef::create({"name"}, decl, implementation_function);
   

4. If the builtin requires a platform dependency (e.g., OpenSSL), use compile-time guards:

   #ifdef REGOCPP_HAS_CRYPTO
     return BuiltInDef::create({"crypto.sha256"}, sha256_decl, sha256);
   #else
     return BuiltInDef::placeholder({"crypto.sha256"}, sha256_decl, Message);
   #endif
   

Removing a Deprecated Builtin

1. Check the deprecated list in BuiltInsDef::is_deprecated in src/builtins.cc.
2. Add the builtin name to the deprecated vector if not already present.
3. Deprecated builtins return RegoTypeError when called, regardless of implementation.

Key Patterns

Argument Unwrapping

// Single type
Node x = unwrap_arg(args, UnwrapOpt(0).type(JSONString));

// Multiple accepted types
Node x = unwrap_arg(args, UnwrapOpt(0).types({JSONString, Int, Float}));

// With function name for error messages
Node x = unwrap_arg(args, UnwrapOpt(0).type(JSONString).func("crypto.sha256"));

// With custom error details
Node x = unwrap_arg(args, UnwrapOpt(0).type(JSONString)
  .func("crypto.sha256").specify_number(true));

Always check for errors after unwrapping:

if (x->type() == Error)
  return x;

Value Extraction

std::string val = get_string(node);            // strips quotes
BigInt ival = get_int(node);
double dval = get_double(node);
bool bval = get_bool(node);

// Optional variants (return std::nullopt on wrong type)
auto maybe_str = try_get_string(node);
auto maybe_int = try_get_int(node);

Result Construction

For scalar results, return bare token nodes:

return JSONString ^ "result";              // string result
return Int ^ BigInt(42);                   // integer result
return Float ^ 3.14;                       // float result
return True ^ "true";                      // boolean true
return False ^ "false";                    // boolean false
return Undefined;                          // undefined (no result)
return err(args[0], "error message");      // error
return err(args[0], "msg", EvalTypeError); // typed error

For compound results (arrays, objects, nested structures), use the rego API helpers declared in include/rego/rego.hh. These handle all Term/Scalar wrapping and cloning correctly, avoiding well-formedness errors:

// Booleans, strings, numbers, null — produce correctly-wrapped Scalar nodes
return boolean(true);                       // same as True ^ "true" but self-documenting
return rego::string("hello");               // note: qualify as rego::string to avoid std::string
return number(3.14);
return null();

// Arrays — items are auto-wrapped via Resolver::to_term()
return array({boolean(true), rego::string("ok")});
return array({header_term, payload_term, rego::string(sig_hex)});

// Objects — built from object_item() nodes
return object({
  object_item(rego::string("key"), rego::string("value")),
  object_item(rego::string("count"), number(42.0))
});

// Nested: array of [bool, object, object]
return array({boolean(false), object({}), object({})});

IMPORTANT: Never manually construct compound result nodes with NodeDef::create(Array), Term <<, Scalar <<, or push_back. These patterns produce nodes that violate well-formedness rules. Always use array(), object(), object_item(), boolean(), rego::string(), number(), and null() instead. These helpers call Resolver::to_term() internally, which handles all wrapping (Term, Scalar) and cloning correctly regardless of whether the input is a bare token, a Scalar, or an already-wrapped Term.

Declaration Types

bi::String, bi::Number, bi::Boolean, bi::Null, bi::Any  // Scalar types
bi::DynamicArray << (bi::Type << bi::String)             // array of strings
bi::DynamicObject << (bi::Type << bi::String) << (bi::Type << bi::Any)  // object
bi::StaticArray << (bi::Type << bi::Boolean) << (bi::Type << bi::String) // [bool, string]
bi::Set << (bi::Type << bi::String)                      // set of strings

Shared Code Between Namespaces

When multiple namespaces share implementation logic (e.g., crypto primitives shared between crypto.* and io.jwt.*):

1. Create a shared internal header: src/builtins/<shared_core>.hh
2. Create a shared implementation: src/builtins/<shared_core>.cc
3. Add the .cc to src/CMakeLists.txt SOURCES
4. Include from both namespace files

Use compile-time backend selection for platform-dependent code:

set(REGOCPP_CRYPTO_BACKEND "" CACHE STRING "Crypto backend: openssl3, '' (disabled)")
if(REGOCPP_CRYPTO_BACKEND STREQUAL "openssl3")
  find_package(OpenSSL 3.0 REQUIRED)
  target_link_libraries(rego PUBLIC OpenSSL::SSL OpenSSL::Crypto)
  target_compile_definitions(rego PUBLIC REGOCPP_HAS_CRYPTO=1 REGOCPP_CRYPTO_OPENSSL3=1)
endif()

Parsing JSON Inside Builtins

When a builtin needs to inspect or validate JSON data (e.g., JWT headers/payloads, JWK keys), always use the Trieste JSON parser (<trieste/json.h>) instead of manual string searching. Manual JSON parsing (e.g., json.find("\"field\""), character-by-character extraction) is brittle and will break on whitespace variations, escaped characters, nested structures, and field-name substrings.

Two JSON AST types exist — use the right one for the task:

For internal inspection, parse into the JSON AST and use json::select with RFC 6901 JSON Pointer paths:

#include <trieste/json.h>

// Parse raw JSON string into JSON AST
Node ast = parse_json(json_str);  // json::reader().synthetic(str).read()

// Field lookup — paths use RFC 6901 format with leading "/"
auto alg = ::json::select_string(ast, {"/alg"});     // std::optional<Location>
auto exp = ::json::select_number(ast, {"/exp"});      // std::optional<double>
auto ok  = ::json::select_boolean(ast, {"/active"});  // std::optional<bool>

// Check field existence (select returns Error node if missing)
Node field = ::json::select(ast, {"/enc"});
if (field->type() != Error) { /* field exists */ }

// Check field type
Node aud = ::json::select(ast, {"/aud"});
if (aud->type() == ::json::Array) { /* it's an array */ }

// Nested paths
auto deep = ::json::select_string(ast, {"/foo/bar/baz"});

CRITICAL: The path argument is a Location initialized from a string literal with {"/field"} syntax. The leading / is required by RFC 6901. Using Location("field") without the / will fail silently.

For return values, use parse_json_to_term() (which runs json_to_rego) to produce Rego-typed nodes suitable for the evaluator. Parse into the JSON AST first for validation, then convert to Rego terms only at the end when building the return value.

For Rego Object nodes (e.g., constraint objects passed as builtin arguments), use try_get_string(node) and try_get_double(node) — these already handle Term/Scalar unwrapping. Do NOT navigate with node / Scalar before calling them.

Testing

OPA Conformance Tests

OPA test cases live in build/opa/v1/test/cases/testdata/v1/<testdir>/. Directory names match OPA builtin names with no separators (e.g., cryptohmacsha256, jwtdecodeverify).

# Run a specific builtin's tests
cd build && ./tests/rego_test -wf opa/v1/test/cases/testdata/v1/<testdir>

# List available test directories
ls build/opa/v1/test/cases/testdata/v1/ | grep <pattern>

# Run all OPA tests (slow)
ctest -R rego_test_opa

Custom Test Cases

Add YAML test cases to tests/regocpp.yaml or tests/bugs.yaml:

- note: mybuiltin/basic
  query: data.test.p = x
  modules:
    - |
      package test
      p := crypto.sha256("hello")
  want_result:
    - x: 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824

Error Message Matching

Error messages must match OPA exactly — conformance tests compare strings literally. When implementing error handling, check OPA's actual error output for the builtin.

Reference Plans

• Crypto & JWT Implementation Plan — Phased plan for implementing crypto.* and io.jwt.* builtins with a shared OpenSSL core