Intro

In FlexPilot, my fork of OpenPilot, I’ve been exploring a new idea for selecting algorithm hyperparameters related to controlling the vehicle. Below is the initial outline of this proof of concept, for the latest information please see the FlexPilot README.

Overview

Multidimensional breakpoints (MDBPs) are a new, highly experimental feature which allows for more complex breakpoints that take into account multiple factors. In other words, you can have multiple sets of traditional breakpoints that are selected by evaluating another set of values. These can be further extended by defining different sources for each dimension allowing you to build highly customizable tunes which weren’t previously possible. As an example: in lat tuning, you can now build a tune that not only takes into account the speed of the car (like we were previously) but also the desired steering angle of the path planner.

Note: at the moment these are only available for indi and steer actuator delay breakpoints. I’m working on adding this functionality to other breakpoints as it makes sense.

Note: In the future, I’d like to add an option to configure the operating mode. This would allow for other ways to evaluate MDBPs such as interpolating the value lists before interpolating the output.

How they work

Much like traditional breakpoints already in FP / OP multidimensional breakpoints also have bp and a v list, only this time they’re a list of lists. Each list inside the bp list is a set of breakpoint points and can represent different factors. What these factors are is defined in another param called the breakpoint source (more below). Each list inside the v list is a set of output values.

Lets take a look at an example tuning the indi inner gain breakpoints:

Example

In this example I’ll walk you through how multidimensional breakpoints work.

Recap

Before we take a look at multidimensional breakpoints lets first look at how normal breakpoints works. Consider the following:

indi_inner_gain_bp: [20, 24, 30]
indi_inner_gain_v: [7.25, 7.5, 9]

Here bp contains a set of v_ego points while v is a set of values for the indi controller. From there the controller would read the car’s v_ego and performs a one-dimensional piecewise linear interpolation, that is: it finds the two closest points in bp to v_ego and then interpolates between the values (in v) at the same indexes as the ones found.

Examples:

  • v_ego = 10, output is 7.25
    • since v_ego is less than the first point (in bp), the first value (in v) is returned.
  • v_ego = 35, output is 9
    • because v_ego is larger than the last point, the last value is returned.
  • v_ego = 22, output is 7.375
    • since 22 is 50% between the first and second points, the output is 50% between the first and second values. (Note: this is generalized to all %s i.e. 10% between points is 10% between values, etc.)

Moving on

Now back to multidimensional breakpoints. As mentioned, multidimensional breakpoints still have points (bp) and values (v) lists. Lets now define what valid multidimensional breakpoints would look like:

indi_inner_gain_bp_multi: [[0, 6], [20, 24, 30]]
indi_inner_gain_v_multi: [[5.25, 5.5, 6], [6, 6.75, 7]]

I mentioned earlier that each list inside of bp was a set of points for some source data but what are they in this case? Well, we need to define another parameter:

indi_multi_breakpoint_source: ['desired_steer', 'v_ego']

Here are our sources for our points (bp): desired_steer and v_ego. More specifically, the first list in bp ([0, 6]) are breakpoints for the desired_steer while the second list are the points for v_ego.

During evaluation, the controller compares the first source (desired_steer) to the closest index in the first list in bp([0, 6]). Once it finds that index it selects the list at the same index from v and interpolates the second source (v_ego) points with the selected values as if it was a single dimensional array. In other words, you can have multiple sets of v_ego breakpoints that are selected based on the desired_steer of the path planner.

In the following lets look at some example evaluations. Here we’ll be evaluating the bp and v arrays above by x. x is the input, it is a list with the same size as the breakpoint source. Each index in x is the value of the breakpoint source at that index. (Example: x: [2, 24] here the desired_steer is 2 while v_ego is 24.)

Examples:

  • x: [2, 24], output is 5.5
    • 1st, compares x[0](2) to bp[0]([0, 6])
    • 2nd, since 2 is closer to 0 select v[0]([5.25, 5.5, 6])
    • 3rd, interp x[1](24) with bp[1]([20, 24, 30]) and v from step 2
      • since 24 is at index 1 of bp[1], index 1 of v[0] is returned
  • x: [5, 35], output is 7
    • 1st, compares x[0](5) to bp[0]([0, 6])
    • 2nd, since 5 is closer to 6 select v[1]([6, 6.75, 7])
    • 3rd, interp x[1](35) with bp[1]([20, 24, 30]) and v from step 2
      • since 35 is larger than the last point in bp[1], the last value in v[1] is returned
  • x: [10, 5], output is 6
    • 1st, compares x[0](10) to bp[0]([0, 6])
    • 2nd, since 10 is closer to 6 select v[1]([6, 6.75, 7])
    • 3rd, interp x[1](5) with bp[1]([20, 24, 30]) and v from step 2
      • since 5 is smaller than the first point in bp[1], the first value in v[1] is returned

Note: this is a simplified and partially wrong / misleading example. For the correction see Param Modifiers (below).

Note: if it’s not clear, step 3 is basically just a 1 dimensional interpolation, like the one shown in the recap, between the second set of points (bp[1]) and the selected set of values (v) from step 2.

Note: once again, I’m planning on expanding this by optionally interpolating both value arrays (i.e. a multidimensional interpolation) in the future.

Breakpoint Sources

Breakpoint sources are another set of parameters in OpParams however these are more of a “meta-param”. In other words, they’re a parameter that describes info about another parameter. More specifically they’re used to know where to look when evaluating the multidimensional breakpoint.

In the examples above, how would x be created? There is a function which takes in the source list and other cereal messages (e.g. CarState, PathPlan, etc.), evaluates each items in the source list, and then selects the source’s value from the correct message (it is during this step that Param Modifiers are applied to the source’s value).

The following should be kept in mind for breakpoint sources:

  1. Sources are predefined and you should only enter existing, valid sources (See BreakPointSourceKeys for an up to date list). At the moment, op_edit does not validate that the sources are correct, you must do this on your own.
  2. The order of sources matters and has a direct affect on how the breakpoints get evaluated. (Exercise for the reader: how do the above examples change if the sources were switched (i.e. ['vego', 'desired_steer'])?)
  3. Currently, this has only been tested using 2 sources. I’m currently evaluating how this feature might work if there were more.

Note: at the moment, these are only being used for multidimensional breakpoints but I’m evaluating if these should be used in other places and or in other ways.

Param Modifiers

Param modifiers are special “keywords” that can be attached to the end of a string param in order to modify its value. See ParamModifierKeys for an up to date list of available modifiers.

Note: at the moment, these are only being used for breakpoint sources but I’m evaluating if these should be used in other places and or in other ways.

Example:

Let’s walk through another multidimensional breakpoint example with param modifiers or rather, let’s rewalk through, and correct, the old one.

Let’s start by defining our initial values again:

indi_inner_gain_bp_multi: [[0, 6], [20, 24, 30]]
indi_inner_gain_v_multi: [[5.25, 5.5, 6], [6, 6.75, 7]]
indi_multi_breakpoint_source: ['desired_steer', 'v_ego']

So far so good, right? Not quite. See, desired_steer is positive when the steering wheel turns counter-clockwise and negative when turning clockwise (in my Toyota Corolla, other cars may be different). Spot the error? All clockwise turns will use the smaller v values even during sharper turns (because negative numbers are closer to 0 than they are to 6).

So how can we fix this? Two ways…

  1. we can update our points (bp) and values (v) to reflect positive and negative angles. This works but it’s kind of a pain to do everywhere and it makes dealing with the param in op_edit harder.
indi_inner_gain_bp_multi: [[-6, 0, 6], [20, 24, 30]]
indi_inner_gain_v_multi: [[6, 6.75, 7], [5.25, 5.5, 6], [6, 6.75, 7]]
  1. we can use the ABS modifier to modify the value of the breakpoint source. In other words, we can take the absolute value of the desired steer from the path planner so that the value is always positive. To do this we append _abs to end of the source like so:
indi_multi_breakpoint_source: ['desired_steer_abs', 'v_ego']

Now that our desired_steer is always positive we can have our points (bp) and values (v) look like this again:

indi_inner_gain_bp_multi: [[0, 6], [20, 24, 30]]
indi_inner_gain_v_multi: [[5.25, 5.5, 6], [6, 6.75, 7]]

With our corrected points and values or corrected sources the breakpoint will evaluate as originally described in the above example.

Final Thoughts

Here are my final thoughts on multidimensional breakpoints:

  • This is still very experimental and mostly still a WIP.
  • I’m very interested in the community’s feedback on this. Not just what I have but also on future mechanics or other general ideas.
  • I’m very interested in seeing what tunes the community can develop with this. The ones I’ve been experimenting with are already well beyond any other previous tunes.

Lastly, here is my current favorite multidimensional indi tune for the TSS2 Corolla:

indi_actuator_effectiveness_bp_multi: [[0, 5], [20, 24]]
indi_actuator_effectiveness_v_multi:[[1.5, 1.75], [2, 3]]
indi_inner_gain_bp_multi: [[0, 6, 15], [20, 24, 30]]
indi_inner_gain_v_multi: [[5.25, 5.5, 6.5], [6.25, 6.75, 8.5], [7.5, 8.5, 10]]
indi_outer_gain_bp_multi: [[0, 3, 7], [20, 24, 30]]
indi_outer_gain_v_multi: [[4, 4.5, 6], [4.5, 5.25, 6.25], [6, 6.5, 7.5]]
indi_time_constant_bp_multi: [[0, 5, 10], [20, 24, 30]]
indi_time_constant_v_multi: [[0.3, 0.5, 1], [1.25, 1.5], [2.25]]
indi_multi_breakpoint_source: ['desired_steer_abs', 'vego']

steer_actuator_delay_bp_multi: [[0], [0, 4, 9, 17]]
steer_actuator_delay_v_multi: [[0.45, 0.4, 0.3, 0.16]]
steer_actuator_delay_multi_bp_source: ['vego', 'desired_steer_abs']

corolla_body_type: 'sedan' (wheelbase: 2.7)
corolla_use_indi: True (tire_stiffness_factor: 0.996)
safetyParam: 50