Best Model Selection

Historical-only and future-required drivers

Keep all drivers in external_regressors. Explain availability in the business instructions; no additional function argument is needed. For example:

Units_Sold is historical-only. From the last observed units, generate units internally at ten percent compounded growth per month. Multiply those units by the supplied future Price_Index for the corresponding month.

The resulting contract records historical_only_predictors = "Units_Sold". Fitting needs observed Units_Sold and Price_Index; forecasting needs Price_Index, Date and the series keys, but not future Units_Sold. If the last observed units are 100 and future prices are 12 and 13, forecasts are 100 * 1.1 * 12 = 1320 and 100 * 1.1^2 * 13 = 1573. Supplied future historical-only values are ignored, not used as a hidden override. Authoring warns when a driver declared historical-only has complete future values, so you can review the contract if those values should be used. A driver the contract still requires in the forecast period must have complete future values in input_data; otherwise authoring stops with invalid_input before any source is generated. Unrelated input columns are also excluded before prediction source runs; only declared future drivers and the required identity columns are passed through. Forecast updates reconstruct the same historical-aware workflow, so newly saved and reloaded updated fits still predict without historical-only columns. Existing saved fits are not rewritten.

Historical-only does not mean carrying forward, imputing, or automatically forecasting a driver. An observed lag must specify its window/date alignment and remain observed throughout the horizon; if it runs out, supply an explicit generation rule or an expected-error example. Internal driver forecasts need a method, starting state and dependency order. Unspecified availability, lag alignment, unsupported horizons, circular dependencies, or contradictory instructions/examples require clarification. Manual mode asks before execution; automatic mode raises a typed error with needs_input feedback rather than inventing business logic. This contract and independent examples cannot prove the correctness of arbitrary prose or generated formulas.

Raw authoring and direct fitting reject missing/nonfinite required history. Existing C1 preparation still zero-fills historical numeric missingness; the model cannot recover the original missingness from those prepared zeros. Models saved without availability metadata retain their original future-driver requirements and content identities. New historical-only models require a package version that supports the declaration.

custom_model_control() has five settings: max_attempts = 3L, validation_budget = 60, package_policy = "installed_finnts", required_properties = character() and forecast_approach = NULL. Build a fresh control to change settings; explicit-setting markers distinguish supplied values from defaults. Supply approval and control by name. Unknown arguments fail. validation_budget is not a hard timeout. Custom-model creation supports one current implementation; obsolete development arguments and artifacts must be recreated, not migrated through compatibility bridges.

In an interactive R session, answer y/yes or n/no, without typing hashes. Case and surrounding whitespace are ignored; blank or EOF declines. details shows readable source and complete examples without executing or regenerating anything. edit collects a requested change and prepares a new review within the cumulative proposal limits. Existing data/capability constraints stay fixed. The default display omits instructions, schema/package inventories and source dumps; a short measured summary follows success. Outside an interactive session, approval = "manual" fails before contacting the LLM; scripts, schedulers and coding agents use approval = "automatic". See ?create_custom_model for the complete contract.

Deterministic Authoring Mechanics

The current authoring contract uses typed structured responses. The default control$package_policy = "installed_finnts" freezes installed direct FinnTS Depends, Imports and Suggests, plus the existing R base/recommended allowance. It does not admit arbitrary installed or transitive-only packages. Use custom_model_control(package_policy = "base_only") to restrict generated code to base R.

FinnTS parses source to derive only the packages actually referenced through pkg::function; the LLM no longer supplies a dependency list. The resulting packages and complete assembled source are hashed before candidate review and authorization. Nothing is installed or attached. Namespace availability and public exports are checked after authorization, before candidate bodies execute. Namespace loading can itself have side effects; it is not a sandbox.

FinnTS also supplies fit/predict signatures, prediction row IDs, and generated example dates, keys, declared-mode coverage and tolerance. The LLM returns calculation bodies, optional helpers, synthetic value arrays, independent expected answers and business rationale. Prediction bodies return finite numeric vectors in request order; the fixed wrapper rejects wrong lengths/types rather than recycling, clipping or imputing. The full reusable model remains ordinary R.

Authoring metadata lists only the workflow’s actual input columns: Date, Combo, Target and declared external regressors. Prediction excludes Target. Internal prepared features such as Date_index.num are not advertised as implicit inputs; an explicitly declared driver with that name remains valid. For a six-observation trend, generated code can construct positions 1:6 and predict at 6 + context$forecast_step, without assuming a prepared feature exists. Only the outgoing metadata copy is filtered. Captured data, saved metadata, accepted examples, approved model identities and older authoring protocols stay unchanged.

For new definitions, FinnTS supplies fit and prediction rows sorted by series and Date, with integer context$forecast_step values derived from each series’ current training cutoff. It restores the caller’s original output order. Each included series must supply its complete forecast horizon; partial-date requests fail before prediction source executes. The runtime does not synthesize missing drivers or infer fiscal calendars. Daily through yearly cadences use their saved period labels. Separate series batches are allowed when the business rule permits them, not as a guarantee that pooled rules are independent. Later refits derive new cutoffs without changing model identity. Saved older definitions keep their original ordering and partial-request behavior. Internal resampling may have shorter scoring windows. FinnTS supplies existing predictor rows through the full horizon, then retains only the original scoring rows. It does not extend training history, expose assessment targets to generated prediction code, fill missing drivers, or change forecast-selection policy.

Body fields prefer calculation statements, but new proposals also accept one complete function with the exact signature function(data, context, parameters) for fit or function(object, new_data, context) for prediction. No defaults, ellipsis, reordered names or extra arguments are allowed in this alternate form. FinnTS extracts that function body once using parsed syntax, before hashing, review and authorization. It never calls the function to discover its result. Equivalent complete-function and body inputs yield the same canonical candidate.

Ambiguous signatures, extra top-level expressions alongside a function, and direct nested function returns produce source_body_format feedback before execution. Local assigned helper functions and callbacks remain valid. Fitted state must still be portable data and prediction must still return a numeric vector. Saved-source replay does not normalize or silently repair old models; their exact source, identity and authorization remain unchanged. This support does not introduce another authoring protocol or any new public argument.

Generated examples are native nested objects and arrays in the structured response, not JSON embedded inside a text field. series remains an array even for one series. FinnTS validates required fields, lengths and captured numeric, character/factor or logical predictor types locally as well. The missing-data policy is fixed to rejection of required missing inputs; the LLM cannot return a blank replacement. This adds neither cleaning nor imputation.

The arbitrary passive fixed-parameter map uses parameters_json to support structured parameter values. Invalid JSON identifies that field explicitly; malformed strings are not repaired by guessing braces or stripping text. Expected answers remain independent of candidate execution. Structural validation does not independently prove their arithmetic. New calls treat LLM-generated numerical expectations as diagnostics: mismatches produce a user warning, not rejection or a request to rewrite the formula. Measured errors remain in the validation evidence, and the warning is retained in model$validation$checks$llm_example_warning. Matching incorrect code and answers can still pass finite comparisons; a warning alone cannot identify which side is wrong.

Bring your own validation examples. Calculate expected outputs with business logic reviewed independently of the generated model, then pass the materialized cases as one table through validation_examples. This is an examples table, not a validation-function callback. Caller-provided output mismatches remain mandatory and can trigger bounded source repair. These exact examples are included in the initial contract request, so the provider can align history requirements and preserve caller error codes/phases before generating source. It must not invent extra business error declarations without matching caller error cases. Include at least two cases, every declared mode, and meaningful edges such as zeros, negative values or calendar boundaries. For example, these independently specified checks describe a three-point mean:

dates <- as.Date(c("2020-01-01", "2020-02-01", "2020-03-01", "2020-04-01", "2020-05-01"))
validation_examples <- data.frame(
  example = rep(c("mean", "edge_zero"), each = 5),
  Date = rep(dates, 2),
  Target = c(10, 20, 30, NA, NA, 0, 0, 0, NA, NA),
  expected = c(NA, NA, NA, 20, 20, NA, NA, NA, 0, 0),
  how = rep(c("(10 + 20 + 30) / 3 = 20", "The mean of three zeros is zero."), each = 5)
)

Use those cases only for that rule and matching cadence/horizon, not for arbitrary custom logic. Caller examples are also provider-visible; keep sensitive real observations out of examples you do not want sent to the provider. Technical failures such as broken R, invalid output shape, nonportable state, required properties and coded-error mismatches still fail and may use bounded repair. Only caller-supplied numerical expectations can request an output-matching rewrite. LLM-generated numerical differences cannot.

Parameters have explicit types and units. Declared numeric weight or decay vectors become atomic R vectors; objects and arrays of records retain list structure. Synthetic numeric history and future columns share double types, including whole-valued historical discounts followed by fractional forecasts. Blank nonnumeric units normalize to not_applicable. Numeric units remain required; no business units are inferred. Malformed declarations receive bounded field-specific diagnostics before contract acceptance.

Required business errors are separate outcomes, not placeholder zero forecasts. An error_policy binds the code and fit/predict stage, and each mode must include valid numerical controls plus tests for its declared errors. Generated error cases use empty expected arrays. Caller error cases set the table’s expected_error column to "predict:zero_denominator" on that example’s rows and leave expected missing. Candidate code raises the portable sibling helper finntsRuleError(code). An unrelated exception or a zero-returning fallback does not pass that test. Additional named error scenarios remain within the existing eight-case limit.

Row-order invariance and fitted-state serialization are mandatory. Other properties are tested when declared: constant forecasts, target-scale equivariance, independent series, fixed history windows, and relative calendars. The caller control$required_properties are frozen hard requirements. Other LLM suggestions are diagnostic only: their failed checks are reported without rejecting valid calculations. This does not weaken caller numerical cases, coded business errors, row identity, finite outputs, portability or exact-version approval. For example, use custom_model_control(required_properties = "constant_forecast") only when constant output is a genuine business requirement, not for every custom model. Calendar tests translate inputs by four years; fixed-calendar rules should not declare that property. Reports retain measured counts and expectation provenance, explicitly labelled caller-supplied or LLM-proposed and unverified. These are behavioral checks, not a general rule compiler or independent arithmetic verifier.

Stateless future-driver formulas may declare zero formula-history need. Their workflow examples still need a historical row to establish the schema and cutoff; FinnTS records that workflow minimum separately. Preprocessing casting failures stop rather than requesting different source arithmetic. Repeating the same source version and failure stops the repair loop without changing fixed answers. Contradictory contracts require a new authoring version and authorization. New source responses may explicitly report contract_conflict; this stops before execution without rewriting the accepted contract or answers. Under the new comparison policy, disagreement with an LLM-generated numerical answer is not a reason to declare a conflict: source should implement the business rule and let validation record any numerical warning. Tagged protocol questions receive package-owned facts within the existing retry budget, while unresolved business questions still require clarification. Automatic defaults cannot override explicitly supplied settings. Invalid defaults declarations are corrected as contract defects, not by changing the caller’s settings. Current-format approved models are not silently revalidated or repaired.

Generated examples use one basic and one edge scenario per declared mode, with a fixed cadence-aligned synthetic cutoff and enough provider-supplied historical values for the rule. Caller-provided examples retain their original tables, values and tolerances. No candidate execution supplies its own expected answers. Those answers and business interpretation remain fallible LLM work: this is not an independent general formula evaluator. New automatic calls supply a percentage-growth convention through the current defaults catalog, but that policy does not prove arbitrary provider prose or code follows it.

The contract requests explicit history_requirements: a nonnegative integer minimum_rows_per_series and unique positive required_lag_periods, bounded by the existing example limits. Offsets use the captured cadence. Rules without exact observed-history lookups may declare an empty lag array. FinnTS validates each example series against the minimum count and every required lag date before accepting the contract. For monthly same-month values and three annual differences, offsets 12, 24, 36 and 48 require four annual observations; seven monthly rows cannot meet that requirement.

Fit-time windows use the minimum count, not forecast-relative lag offsets. For example, “use the median of the last six actual months for every forecast month” needs minimum_rows_per_series = 6 and required_lag_periods = []. The model computes that median at fit time and repeats it across the horizon. Declaring lags 1 through 6 instead asks for observations before each future forecast date. With three forecast months, offsets 1 and 2 eventually point into the unobserved forecast window. More old observations cannot fix this.

Repair feedback explicitly identifies lag_reference = "forecast_date" and the conflicting unobserved_lag_periods. Exhaustion errors include the same bounded explanation. A provider must correct an unaccepted declaration that misrepresents the rule; FinnTS never silently drops its lags or changes expected answers. Seasonal forecast-date lookups retain their existing strict checks.

Missing coverage is rejected during contract review, before source generation. Authoring can retry an unaccepted history declaration that conflicts with structurally valid caller examples. For example, a fit-time growth window must not be described as observed lags reaching into the forecast window. The next attempt receives numerical-case row counts and impossible-lag feedback; intentional error cases are excluded from those counts. The declaration must still express the actual business rule, not weakened history needs merely to pass. Exhaustion returns an error under the existing contract-attempt limit. Only unaccepted generated examples may be replaced. Caller examples, accepted answers and the source-repair oracle never change, and no history is invented, imputed or silently extended. The declaration is testable metadata, not a proof that the LLM has interpreted every possible business rule correctly.

New source preflight also rejects obvious uninitialized protocol-state references, such as object$history <- data in fit when object was never created. Fit receives data, context and parameters; prediction receives object, new_data and context. The check never inserts an object or rewrites a formula. It inspects ordinary evaluated syntax and lexical bindings without execution, while leaving quoted/formula and unknown nonstandard-evaluation syntax alone. Uncertain control flow still requires runtime validation. Bounded source_binding evidence names the function and symbol for source repair, without arbitrary exception text.

Approved models never gain new permissions on forecast reuse. Unknown protocols, altered scaffolds and changed runtime/package metadata are rejected.

Automatic Authorization

Set approval = "automatic" to authorize generated R and bounded repairs without manual review. It never calls readline() or requests clarification: it returns a validated model or a typed actionable error. This deliberately gives up source review, not numerical validation. The code runs in the current R session with your permissions, without a sandbox or reliable interruption.

custom <- create_custom_model(
  instructions = "Use the mean of historical targets independently for each series.",
  llm = llm, input_data = data_tbl,
  combo_variables = "id", target_variable = "value",
  date_type = "month", forecast_horizon = 3,
  approval = "automatic"
)

Automatic calls use one closed, package-owned defaults catalog. Explicit supported instructions and supplied metadata take precedence; incompatible or contradictory requirements fail instead of silently widening capabilities.

Never invent business offsets, multipliers, numeric growth rates, allocation percentages, caps, commercial targets, required weights/drivers/formula terms, unspecified fiscal calendars or aggregation semantics. If essential context remains unresolved, start a new call with explicit instructions and compatible metadata. The catalog does not infer these choices from observed values or improve the requested rule.

For a prior-year seasonal value plus average growth over three years, four annual observations of 100, 120, 150 and 180 give percentage changes of 20%, 25% and 20%. Their equally weighted arithmetic mean applied to 180 gives 219. This is not CAGR, an absolute increment or extra compounding across forecast steps. The requested lookback and seasonal/calendar alignment still govern which observations are used. Undefined division still errors; no epsilon, clipping or fallback rate is introduced. Existing absolute-growth worked examples and caller-supplied answers are never rewritten to fit this convention.

The fixed contract records catalog_version, effective settings and actually applied defaults. The approved model retains their JSON in custom$validation$checks$automatic_defaults; success prints the applied choices and measured results. Semantic defaults-used declarations come from the provider and are restricted to catalog keys/values. Neither those declarations nor finite examples prove arbitrary prose/code semantics for every future input. growth_basis is recorded only when the provider declares that the percentage fallback was applied, not for explicit growth bases or non-growth rules.

Every source version receives a separate automatic authorization before execution. The record has intent_confirmed = FALSE, allow_code = TRUE and mode = "automatic", bound to its version_id; it never claims human review. Manual envelopes record the three explicit confirmation fields. Definition hashes and Model_ID do not change solely because approval policy differs, while run and Agent provenance distinguish the policies. Forecast reuse does not call the authoring LLM or regenerate source.

The only public approval values are "manual" and "automatic". Explicit NULL, "interactive" and "deferred" fail. Manual approval outside an interactive R session fails; use "automatic" there.

Example Dates and Retries

Every worked example must forecast exactly the next forecast_horizon consecutive cadence dates immediately after its last historical Date. With monthly horizon 3 and history ending December 2025, those dates are January-March 2026. A prior-year seasonal rule uses January-March 2025 values from history; it does not move the forecast window another year forward. History must contain the observations the rule needs. For a fixed 11% increase, prior-year values 10, 20 and 30 produce 11.1, 22.2 and 33.3. Zero and negative values remain valid unless explicitly excluded by the supported rule.

Examples must cover every declared model_type and no undeclared modes. A local-only contract needs only local examples, with exactly one series per example; it must not add a global example merely because FinnTS supports global models. A global-only contract needs only global examples, each with at least two series. Either single-mode contract still needs two to eight total examples, including an edge case. Mixed contracts require coverage of both declared modes.

An undeclared example mode produces specific mode feedback during contract retries. FinnTS does not silently remove or relabel the example or enable another mode to make it pass. The provider may regenerate the unaccepted proposal within the existing attempt limit; caller-provided examples must instead be corrected by the caller. Valid tolerance values are not reported as the cause of a well-formed example’s undeclared mode.

When a generated contract is rejected, the next bounded contract attempt receives safe package-owned feedback explaining the rejection category and correction. The final error includes the last safe rejection if attempts are exhausted. Raw exception messages, private values, paths and provider output are not copied into that feedback.

Feedback also names the rejected field, such as parameters_json or examples.series. Unexpected validator failures stop rather than consume more provider retries. An optional diagnostics$rejected_contract snapshot retains only known provider-visible response fields, their hash, the field/code and an omission marker. It shares the existing one-MiB diagnostic cap; oversized content is omitted without truncating an accepted oracle. Raw exceptions, Chat objects and actual training data are excluded. This snapshot is diagnostic evidence, not an accepted contract or authorization. No extra automatic file is written.

Automatic business_clarification errors retain the unanswered questions at diagnostics$rejected_contract$proposal$questions, subject to the same size limit. The normal error message also displays the questions, labelled as provider clarification rather than independently verified input errors. Each question is limited to 500 displayed characters with a truncation marker, and control characters are removed; the original diagnostic text is unchanged. No debugger-frame search is needed to see ordinary questions. Supplied metadata and catalog defaults should resolve known choices; genuinely missing business terms still require clarification rather than guessed answers.

For insufficient declared history, retries receive feedback$history with package-computed minimum rows, the supplied row counts per scenario/series, declared lags and whether all lookups can remain historical through the horizon. Generated cases use scaffold identities; caller cases use their already provider-visible names and series keys. Only counts and identifiers are included in this feedback, not observed training values. No rows or answers are filled or rewritten. New examples should use simple, independently checkable arithmetic; source repairs cannot replace a bad accepted oracle merely to pass validation.

The provider may replace invalid generated examples only before the contract is accepted. Caller-provided examples are never replaced: structurally invalid supplied examples fail when checked, without spending further provider attempts on them. A conflicting unaccepted history declaration is retried instead of blaming or changing a valid caller example; earlier protocols retain their existing history-failure behavior. After contract acceptance, code repairs must preserve its examples and tolerances. The existing date checks, numerical checks, three-attempt limits and manual or automatic authorization policy remain unchanged. Corrective feedback improves repair information; it does not guarantee successful live generation.

Source Repair Diagnostics

New source requests contain fit_body, predict_body and optional named helper functions. Body fields may use the exact complete-function alternate form above. FinnTS assembles the fixed interfaces and row-identity wrapper from parsed expressions, without evaluating them. A passive preflight checks syntax, permitted namespaces and resolvable base/lexical function calls, without guessing packages for unresolved calls. Units come from the accepted interpretation and examples: a lag of twelve monthly observations means one year, not twelve years. Parameters needed by predict() belong in fitted state.

Dependency repairs also receive bounded export-location hints from already-loaded eligible namespaces. FinnTS neither loads additional namespaces for this lookup nor substitutes functions automatically; identical exported names may have different semantics. Date lookup guidance preserves matching key classes: vapply() can drop a Date result’s class even with a Date template. These aids reduce mechanical errors, but do not prove arbitrary generated logic correct.

Every custom model uses one current calendar-helper bundle. The reviewed helper source includes:

• finntsShiftDate(dates, periods, date_type, invalid_day = "error") returns Date values in the supplied order for integral day, week, month, quarter or year offsets. Negative offsets look back; offsets must be scalar or match the dates’ length. Nonexistent month-end or leap-day dates error by default. invalid_day = "last_day" is available only when the business rule explicitly permits clamping. The helper does not infer fiscal calendars.
• finntsMatchDate(dates, history_dates, combo = NULL, history_combo = NULL) returns historical row indices, preserving request order. Pooled histories must supply both character series-key vectors. Date vectors must retain their class, and historical keys must be unique within each series. Missing keys return NA rather than filling history or changing the model’s error policy.
• finntsRowsDate(dates, history_dates, history_combo) returns a list of all matching historical row indices for each requested date. Repeated dates across different series are valid; duplicate series/date keys still error. Missing dates return integer(0). The LLM implements the actual aggregation, weighting, grouping and business missing-history behavior.

These helpers handle calendar mechanics, not the business calculation. Their source and the reserved finntsDateError helper are frozen into model identity, repair snapshots and saved approvals. They require no new package dependency or provider call during prediction and leave room for 24 provider helpers within the unchanged source-size limit.

New global contracts distinguish shared fitting from prediction scope. prediction_cohort declares series, groups with explicit stable group columns, or all fitted series. A shared regression can predict complete series batches independently; portfolio allocations generally need the full cohort. FinnTS checks required peers and group membership before generated prediction code runs. Resampling expands execution to required peers and dates while retaining the original analysis rows and scoring only original assessment rows. Local models use series scope; global models declare the scope their rule needs.

Explicit global authoring checks every supplied series, then validates generated code on a deterministic representative sample of at most 100 series within 10000 historical rows, which keeps authoring preparation fast on large cohorts. The sample always includes edge series (largest and smallest scale, most zeros, negatives, flattest, shortest), covers multi-variable combo levels when they fit, and fills the rest in hash order. Its counts, never series labels, are disclosed to the provider; forecasting later fits the approved model on every series. Regression guidance asks generated code to retain feature order, category levels, contrasts, scaling and time origin in portable fitted state. These mechanics do not choose formulas: the LLM still writes the actual model, transformations and allocation.

A genuine fixed-contract contradiction stops authoring. An explicitly reported implementation difficulty can use only remaining source attempts; it cannot change examples, extend the retry budget or bypass exact-version approval. Provider failures are separate operational errors with bounded category, HTTP status when known, phase, prompt bytes and elapsed time. Unknown causes remain unknown; raw provider messages, response bodies and headers are not retained.

The shared source guard catches explicit Date-template vapply() and plainly Date-returning sapply() simplification before execution. It preserves numeric iteration, quoted code, local shadows and explicit Date/numeric conversions; unknown expressions are not treated as proven type errors. Runtime helper conditions distinguish date_key_type, date_key_duplicate, and date_shift_invalid from genuine missing-history business errors. Together with date_class_loss, these produce bounded technical repair feedback without exposing lookup values, rewriting examples, increasing retries or bypassing approval. They are not a general proof of generated-code correctness.

Unsupported development formats require recreation, not compatibility bridges. Approved models never acquire helpers silently. Approved helper source remains unchanged during later-cutoff refits, serialization and worker execution, just like the model’s calculation source.

Repairs receive the previous source and a version-bound failure record identifying the phase, reason and, where applicable, worked-example index and mode. Fitting, prediction, arithmetic mismatch, nonportable state, roundtrip disagreement, holdout failure and elapsed-budget failure remain distinct. Up to four keyed expected/actual comparisons may be provided for fixed worked examples. Real holdout values, driver values and business-series identifiers are never added.

Recognized missing-object runtime errors now use reason = "unbound_symbol" with a separate failure$unbound_symbol value, such as history. That value must be an ASCII identifier of at most 120 bytes referenced in parsed candidate code. Names found only in strings, member names, assignment bindings or quoted/formula expressions do not qualify, and sensitive-looking identifiers are withheld. Unrecognized or localized error wording remains generic. No raw exception text, stack trace or runtime value is included in this diagnostic.

The source request instructs prediction code to bind fitted-state aliases explicitly. For example, history <- object$history makes history$Date valid; with(object$history, ...) alone exposes the table’s columns but does not create the history alias. Valid data masks and nested functions remain supported. FinnTS reports runtime evidence rather than pretending to infer every R scope, and never injects an alias or changes the generated calculation automatically.

Manual repair reviews and exhausted-attempt messages show the bounded missing identifier. Repairs still preserve the accepted contract, caller examples and attempt budget; manually repaired source still requires fresh version-specific consent. Saved failures without the new field and existing approved models remain readable without changing their source or triggering new authoring.

Messages are package-owned by default. An exact constant literal stop() message already present in source can be returned only after restrictive length/content checks. Dynamic error values, arbitrary stack traces and credential/path-like content are withheld. Preparation, missing dependencies and RDS I/O remain hard operational failures instead of being disguised as repairable business logic.

New-protocol dependency failures carry up to eight static references and 32 eligible package names within eight KiB, with explicit omission counts. They distinguish undeclared/unavailable packages, unexported/private members, forbidden operations and unresolved functions. For example, an eligible package can still have a requested function that is not exported. Repairs receive that exact reference, not advice to install the package or change the accepted formula. Call arguments and arbitrary namespace-load errors are never included.

Capture an authoring error to inspect its retained diagnostics without entering the debugger:

result <- tryCatch(
  create_custom_model(
    instructions = "Use the same calendar month in the prior year.",
    llm = llm, input_data = data_tbl,
    combo_variables = "id", target_variable = "value",
    date_type = "month", forecast_horizon = 3,
    approval = "automatic"
  ),
  error = function(error) error
)
feedback <- custom_model_feedback(result)
print(feedback[c("status", "stage", "code", "next_action", "attempts", "questions")])
if (identical(feedback$status, "approved")) custom <- result

Feedback schema 1 covers approved, review_required, needs_input, and failed. It includes available candidate/contract/request identities, resolved inputs and bounded safe details without dumping source, training rows or raw exception text. Questions are labelled provider proposals. NULL counts mean unavailable evidence, not unlimited retries; approved models do not retain attempt histories. needs_user_input is true for questions and manual review. Preflight failures without a model have unknown approval policy (NULL); known error stages are retained. Errors still raise normally and are never usable models. Full diagnostic history remains separately available in result$diagnostics when an error is captured. Both reports can contain confidential column names, provider-visible questions or explicitly chosen artifact paths.

For code = "invalid_error_contract", inspect feedback$details$error_contract. Its reason distinguishes missing_error_example, undeclared_example_error, error_phase_mismatch, invalid_error_declaration, and missing_numerical_control; bounded fields identify the error code, expected/actual phase, mode and example index when known. For example, a rule declaring fit:zero_units needs both an independent error-outcome case with that exact phase/code and a valid numerical control. A predict:zero_units example is a phase mismatch, not an acceptable substitute. Supply the missing independent case when requires_caller_input is true. With fixed validation_examples, a declared error that none of them covers is first returned to the provider within the attempt budget, because the provider may have added an error the rule never required; if the declaration remains after the budget, feedback is needs_input. Caller examples that lack a numerical control for a declared mode cannot be repaired by the provider, so that case returns needs_input immediately. Package-generated questions are labelled separately from provider proposals. Repairable generated examples receive these same details within the existing attempt budget; exhausted attempts retain them. Caller expectations, tolerances and error outcomes are never invented or changed during repair.

Coding agents should relay needs_input questions and any review request to the user, then call create_custom_model() again with refined instructions or approval = "automatic". Feedback does not authorize execution, restart an interrupted action or increase budgets. LLM configuration remains per call; no global provider configuration or hard provider/execution timeout is introduced.

The bundle retains at most 12 compact attempts and one candidate/accepted-contract snapshot under a one-MiB JSON limit. Oversized snapshots have an explicit omission flag and retained hashes; validation examples themselves are never truncated. The condition carries diagnostics in memory; no data persistence or cloud upload occurs. Treat source, instructions and provider-visible examples as confidential. The snapshot excludes real training rows, fitted objects and Chat.

Source repair cannot change the rule, units, parameters, expected answers or tolerances, and each candidate still needs exact manual or automatic authorization. The three-attempt limits remain unchanged. Tests cover last-value, mean/offset, rolling and weighted means, seasonal lookup/growth, explicitly defined mean YoY growth, driver ratios and conditional rules, plus quarterly and pooled controls. These are examples of supported ordinary-R logic, not a closed model menu or a promise that every arbitrary or ambiguous request succeeds. Live-provider reliability requires separate evaluation; no sandbox or hard timeout is added.

Manual Review

Manual approval runs only in an interactive R session, using the console replies described in Experimental custom models. Business questions are asked in the same console. Changed source always requires a fresh review; expected examples stay fixed during code repairs.

Scripts, scheduled jobs and coding agents run noninteractively. Manual approval there fails before contacting the LLM; use approval = "automatic" instead. Coding agents should show the user the instructions and example table they plan to submit, and relay any authoring error back to the user.

Authoring validation uses the currently loaded FinnTS functions in the current R session, including devtools::load_all(). It starts no child process and installs no package snapshot. Load the intended development code and revalidate after changing it; FinnTS does not compare loaded functions against on-disk source. Local authoring prepares at most three deterministic series; explicit global authoring prepares a representative sample of at most 100 series. Both enforce at most 10000 total historical rows, using a separate temporary root for raw data. Existing prepared runs are read without writes. Explicit hierarchy authoring samples up to three raw leaf series and retains every derived node; a prepared hierarchy is read completely. The 10000-row limit applies to all historical node rows, and larger hierarchies are rejected rather than truncated. The configured provider receives instructions, column types and fixed examples during generation, not automatically raw Target/regressor values or business series labels. The return is passive and serializable; it contains measured summaries but no fitted state, raw data, callbacks or Chat.

Generated R runs in your current session with access to session/environment values and your account’s filesystem/network permissions. There is no sandbox, crash isolation or reliable automatic interruption; stuck code may require restarting R. control$validation_budget is an elapsed acceptance budget checked after evaluation returns, not an interrupting timeout. Ordinary options, RNG, working directory and library paths are restored on normal return/error, but arbitrary global, environment, filesystem or native-code side effects cannot be rolled back. Manual mode requires review before consenting; automatic mode explicitly waives that review. Sample validation is evidence, not a proof of all future behavior. No packages are installed and authoring does not register the result or publish a forecast. Select its alias explicitly through models_to_run when supplying custom_models = list(business_mean = custom) to standard preparation.

Custom models train on a dedicated raw recipe, C1. Add it to recipes_to_run (for example c("R1", "C1"), or "C1" for a custom-only run). prep_data() writes the C1 files alongside any R1/R2 files, so built-in and custom models can share one run without sharing a representation:

A prepared C1 recipe with no selected custom model is ignored with a warning. forecast_time_series() exposes stationary, retaining its existing TRUE default for ordinary calls.

Custom workflows preserve Date and Combo plus explicitly declared predictors. They do not acquire implicit PCA, feature selection, target transforms or hidden features. Missing required data is an error. Local/global declarations constrain the enabled run modes; a definition with no compatible enabled mode fails rather than enabling a different mode. Custom fits use chronological analysis folds and fixed parameters, without tuning, source regeneration or package installation. Custom source or incomplete-prediction failures cannot be silently dropped to make a mixed run appear successful.

C1 retains eligible original numeric, character, factor, logical and Date columns; reserved outcome/assessment columns and unsupported types are excluded. Supply complete history for every declared custom driver, and future values through the forecast horizon for prediction-required drivers. A driver explicitly declared historical-only in the reviewed contract needs no supplied future values; its future use must follow the agreed historical rule. Finn neither fills missing prediction-required drivers nor replaces them with zero. This remains true when built-ins use other xregs, the Agent’s first iteration uses none, or update recovery uses simple defaults. C1 source identity is pinned before recipe writes, including partial preparation: changing eligible source values requires a new standard run or Agent version, not a restart.

A manifest attached to the normal -model_workflows.rds artifact pins the effective pool, exact source content, attestations and preparation context. Standard preparation needs no separate custom registry or pool-manifest file. Reordered equivalent selections can reuse it; changed definitions, approvals or context require a new run. Custom Model_Name values keep their readable aliases, including underscores. Custom Model_ID values use custom-<full content hash>--<local|global>--C1, avoiding ambiguity with existing average-component separators. Saved workflows, fits, predictions and averages are checked before completion shortcuts; stale definitions and foreign components raise errors instead of being silently repaired.

The ordinary accuracy and plausibility policy still selects the best eligible candidate. With average_models = TRUE, custom forecasts can be averaged with other selected models, and learned ensembles can use custom back-test forecasts. Enrollment does not guarantee selection. Exact single-rule workflows must explicitly disable averaging; bottoms_up alone does not do that. Supported storage backends retain their ordinary access and package requirements.

Reviewed Hierarchy Rules

Use create_custom_model(control = custom_model_control(forecast_approach = "standard_hierarchy"), ...) or "grouped_hierarchy" to review a new local-per-node definition. Its contract explicitly permits normal reconciliation to adjust the rule’s base predictions before publishing the bottom level. An old bottom-up approval cannot be silently upgraded; changing hierarchy semantics requires a new version and human approval.

Forecast enrollment must include both compatible custom rules and eligible FinnTS models in one shared models_to_run pool. A custom-only hierarchy request errors without adding a model or switching the approach. Individual rule validation during authoring does not require a competitor and does not enroll a forecast. Different winners at different nodes, including custom winners everywhere, are valid outcomes of an eligible mixed pool.

This profile is local and univariate; custom nodes train on original-scale C1, while built-ins keep their usual settings. Simple averages and learned ensembles are allowed. Every candidate must cover every required node and forecast key. Missing coverage cannot be replaced by snaive, zero-filled or silently omitted. Existing hts mathematics and quality selection are reused. Per-series routing, pooled/global hierarchy and Agent hierarchy remain later work. See the hierarchy walkthrough for explicit controls and the compatibility evidence matrix.

Custom models in Agent iterations and updates

Supply approved objects with set_agent_info(custom_models = list(business_mean = custom)). The setup models_to_run argument is removed: built-in candidate search remains the Agent’s responsibility, while every mode-compatible enrolled custom model is mandatory from the first iteration onward and executes before built-ins. The LLM cannot omit these models or alter their approved logic. Required drivers and raw C1 are retained; ordinary built-in transformations, ranking and average-model accuracy signals remain intact. Custom-only forecasting without built-in search belongs in a standard run.

Each version’s existing CSV log always contains one custom_models names column, using NA when none are enrolled. Setup saves one complete, immutable enrollment bundle at prep_models/<project-hash>-<hash-of-agent-run-id>-custom-models.rds. The returned Agent object and dispatched worker metadata contain names/references, not untrained custom workflows or source/approval payloads. Workers load the exact version-owned bundle after dispatch. Sequential, local-machine and Spark dispatch are supported with CSV data, RDS model objects and no inner parallelism for this Agent profile. Spark still requires its ordinary mounted-storage and worker-package setup; permitting dispatch is not evidence of a live-cluster test.

Compatible updates require an identically enrolled new parent, unchanged cadence, regressors and enabled modes, and explicit allow_iterate_forecast = FALSE. The positive whole-number forecast horizon may change, subject to the approved rule’s history and driver requirements. Successful series replay their saved winners and component subsets. New series and recoverable built-in fitting or forecast-quality failures use Finn’s existing default-local fallback, with all compatible current-version custom models added first, plus enrolled foundation models whose credentials are available. The coordinator checks foundation availability once and passes it to workers for consistent default-model validation. Changed global cohorts are refitted using compatible current peers rather than reusing a pooled fit from different membership. Saved global winners are published only for surviving series; new peers still receive default-local forecasts. Required custom drivers remain mandatory. Custom rules are never retuned or regenerated; mixed retuning reuses their fixed results.

Storage/authentication failures, missing or malformed required artifacts, enrollment/identity mismatches and custom-source failures remain hard errors. Fallback cannot silently drop a custom model or fabricate successful output. Accepted defaults retain provenance and completed restarts do not refit them. Update replay provenance uses a separate logs/...-agent-custom-update.rds record: it identifies the predecessor and saved choices, not another copy of the untrained enrollment bundle. Older unreleased custom artifact layouts must be recreated with the current setup/preparation functions; they are not migrated.

Least-issues fallback

Standard and iterative forecasts, and fresh default forecasts for both new series and failed updates, can select a fallback from existing complete, finite predictions with usable backtest accuracy. Missing, extra or duplicate required keys, nonfinite predictions, unavailable accuracy, and artifact/provider failures cannot be waived. If no usable candidate exists, the run still fails.

Fallback ordering is the number of distinct failed plausibility checks, maximum normalized failure severity (including magnitude beyond the existing bound), backtest WMAPE, then stable model ID. Unlike normal selection, this last-resort ranking is not restricted to the accuracy shortlist. Applicable soft checks are assessed even on usable extreme forecasts, so a previously skipped zero risk cannot make an extreme forecast look safe. Normal successful selections and their seasonal-fidelity preference remain unchanged.

The selected candidate receives the existing Best_Model = "Yes" flag. A catchable finnts_forecast_selection_fallback warning names the series, model, and concerns; parallel workers relay this warning to the caller. No extra models are fitted, no predictions are fabricated, and strictly rejected components are not introduced into averages or learned ensembles. Fallback evidence stays in memory: there are no new saved columns, status values, attributes, or tracking files. Existing forecast, accuracy and best-run values reflect the actual selected result.

Updating an earlier update

Some older hierarchical global updates saved fitted models and reconciled forecasts, but no per-series source forecasts. When all expected source artifacts are genuinely absent, Finn can reuse that run’s actual saved global model/recipe components. One component is used directly; multiple components are averaged equally at every source node. A warning identifies this policy once per global group. Requested models without saved fits are not added, and recipe identities are preserved rather than replaced with R1.

This compatibility path requires valid reconciled output covering the global series being updated and consistent saved model settings and fits. Each requested series must have unique, complete future and backtest keys matching the saved preparation calendar and train/test splits, including every day when weekly forecasts were expanded to daily output. Missing or unreadable preparation is a hard artifact error. Unrelated local winners’ rows are not required. It cannot recover heterogeneous historical choices from reconciled Best-Model rows. It does not search older Agent versions. Partial source sets, existing empty or corrupt source content, ambiguous winners, and storage/access failures cannot be treated as legacy absence. A missing optional unselected average remains valid; a schema-correct empty average represents an intentionally cleared average, not proof that the entire source set is absent.

Accepted new hierarchical updates save their pre-reconciliation components and selected averages in the existing per-series format. Unused average files are overwritten with empty tables so an interrupted refit cannot leave a second winner behind. Publication verifies readable source coverage, selected identities, fitted components, prediction keys and average arithmetic. Restart checks compare each node’s intended selection, not just the shared model pool, unless hierarchical recovery is enabled. A changed choice must then match recovery recomputed in memory from the saved source predictions, prepared history, and predecessor selections, without fitting or reconciliation. Existing source artifacts record the per-node mapping for subsequent updates. No additional tracking columns or values are stored in run logs, parent best-run records, or final run metadata. Source files are authoritative: missing or ambiguous winners and invalid components are rejected, but there is no separate provenance record to authenticate an internally consistent flag edit. The withdrawn global-update tracking columns and default_reforecast_status, if encountered in older metadata, are omitted when that metadata is next written; existing files are not migrated in place. If the raw hierarchy table omits a historical target, hierarchical global updates use the corresponding original-scale prepared source history already used for selection. Observed targets stay unchanged, and future targets remain unknown. This keeps delivered accuracy and completion logging usable for sparse series without inventing actuals or bypassing the accuracy gate. The updater requires the logger to confirm every requested global series before reporting completion.

Changing hierarchy membership

Selection recovery uses the predecessor’s identities, while reconciliation requires the current hierarchy. Removed bottom series and orphaned aggregates are excluded; input ordering or an unchanged node count does not establish identity. Generated HTS aggregate labels can be reassigned when membership or hierarchy ordering changes. Finn compares their membership on surviving bottom series: a reassigned label is treated as a new node, not as the old aggregate. Added or reintroduced nodes have no selection in the immediate predecessor and follow the default-local workflow. Their saved local result is the predecessor for the next update. This includes first/last group members and overlapping grouped aggregates, not only bottom-level additions.

For an inner global hierarchy, new source nodes inherit the predecessor’s selection when every previous source used the same single model or average. This includes the uniform selection recovered from older reconciled-only runs. Finn retains all existing choices and saves the new nodes’ component and selected average forecasts, allowing the existing global series to remain global through later updates. New source predictions must pass the same completeness, component and quality checks; reassigned-label safeguards remain in force.

This source-node coverage does not promote a newly added bottom series to a global winner: its default-local delivery routing remains unchanged. When previous source selections differ, new inner sources are evaluated from the already-refitted pool rather than inheriting another node’s identity. The whole inner hierarchy is rejected only when it cannot supply a complete hard-valid source set or a surviving generated label was reassigned. Pre-expanded hierarchy inputs retain their local/global per-node routing, followed by one outer reconciliation. The existing new-series limits, quality gates, retuning rules and coherent-global-run policy remain unchanged.