inv_token.h File Reference

Token vocabulary and structures for foundation-model tokenization of InvSourceEstimate. More...

#include <cstdint>
#include <vector>

Include dependency graph for inv_token.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
struct	INVLIB::InvToken
	One element of a tokenised neural-source representation. More...
struct	INVLIB::InvTokenizeOptions
	Tokenization options controlling layer inclusion and sub-sampling. More...

Namespaces
namespace	INVLIB
	Inverse source estimation (MNE, dSPM, sLORETA, dipole fitting).

Enumerations

enum class

INVLIB::InvTokenId : int32_t {   INVLIB::Pad = 0 , INVLIB::Bos = 1 , INVLIB::Eos = 2 , INVLIB::Sep = 3 ,   INVLIB::MetaBegin = 10 , INVLIB::MetaEnd = 11 , INVLIB::GridBegin = 20 , INVLIB::GridEnd = 21 ,   INVLIB::GridRow = 22 , INVLIB::CouplingBegin = 30 , INVLIB::CouplingEnd = 31 , INVLIB::GroupBegin = 32 ,   INVLIB::GroupEnd = 33 , INVLIB::FocalBegin = 40 , INVLIB::FocalEnd = 41 , INVLIB::DipoleBegin = 42 ,   INVLIB::DipoleEnd = 43 , INVLIB::ConnBegin = 50 , INVLIB::ConnEnd = 51 , INVLIB::ConnEntryBegin = 52 ,   INVLIB::ConnEntryEnd = 53 , INVLIB::PosBegin = 60 , INVLIB::PosEnd = 61 , INVLIB::MethodUnknown = 100 ,   INVLIB::MethodMNE = 101 , INVLIB::MethodDSPM = 102 , INVLIB::MethodSLORETA = 103 , INVLIB::MethodELORETA = 104 ,   INVLIB::MethodLCMV = 105 , INVLIB::MethodDICS = 106 , INVLIB::MethodSAM = 107 , INVLIB::MethodMixedNorm = 108 ,   INVLIB::MethodGammaMAP = 109 , INVLIB::MethodDipoleFit = 110 , INVLIB::MethodRapMusic = 111 , INVLIB::MethodPwlRapMusic = 112 ,   INVLIB::SpaceUnknown = 150 , INVLIB::SpaceSurface = 151 , INVLIB::SpaceVolume = 152 , INVLIB::SpaceMixed = 153 ,   INVLIB::SpaceDiscrete = 154 , INVLIB::OrientUnknown = 170 , INVLIB::OrientFixed = 171 , INVLIB::OrientFree = 172 ,   INVLIB::OrientLoose = 173 , INVLIB::Amplitude = 200 , INVLIB::Vertex = 201 , INVLIB::TimeVal = 202 ,   INVLIB::FreqVal = 203 , INVLIB::PosX = 204 , INVLIB::PosY = 205 , INVLIB::PosZ = 206 ,   INVLIB::MomX = 207 , INVLIB::MomY = 208 , INVLIB::MomZ = 209 , INVLIB::Goodness = 210 ,   INVLIB::ChiSquared = 211 , INVLIB::Correlation = 212 , INVLIB::ConnValue = 213 , INVLIB::GridIndex = 214 ,   INVLIB::ValidTrue = 250 , INVLIB::ValidFalse = 251 , INVLIB::DirectedTrue = 252 , INVLIB::DirectedFalse = 253 ,   INVLIB::NSources = 280 , INVLIB::NTimes = 281 , INVLIB::NGroups = 282 , INVLIB::NDipoles = 283 ,   INVLIB::NMeasures = 284 , INVLIB::NIndices = 285 , INVLIB::NFreeDof = 286 , INVLIB::TStep = 287 ,   INVLIB::MeasCoh = 300 , INVLIB::MeasImCoh = 301 , INVLIB::MeasPlv = 302 , INVLIB::MeasPli = 303 ,   INVLIB::MeasWpli = 304 , INVLIB::MeasGranger = 305 , INVLIB::MeasPdc = 306 , INVLIB::MeasDtf = 307 ,   INVLIB::MeasCorrelation = 308 , INVLIB::MeasCrossCorr = 309 , INVLIB::MeasOther = 310 , INVLIB::QuantBinBase = 1000 }

Functions
std::vector< int32_t >	INVLIB::tokenIds (const std::vector< InvToken > &tokens)
std::vector< float >	INVLIB::tokenValues (const std::vector< InvToken > &tokens)

Detailed Description

Token vocabulary and structures for foundation-model tokenization of InvSourceEstimate.

Author

Christoph Dinh chdin.nosp@m.h@nm.nosp@m.r.mgh.nosp@m..har.nosp@m.vard..nosp@m.edu

Since

2.0.0

Date

June, 2026

LICENSE

Copyright (C) 2026, Christoph Dinh. All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
• Neither the name of MNE-CPP authors nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

The token vocabulary defines a self-describing sequence format for serializing multimodal neural source representations into a flat token stream that transformers and other foundation models can process.

Each token carries an integer vocabulary ID and an optional continuous floating-point value. Structural tokens (section delimiters, metadata labels) use only the ID; value-carrier tokens (amplitudes, positions, times, indices) pair the ID with the continuous quantity. This hybrid design supports both:

• Models with continuous-value embedding layers (use id + value)
• Purely discrete language-model architectures (use id only, apply external quantization to the value field before embedding)

A typical token sequence for a surface MNE estimate has the form:

[BOS] [META_BEGIN] [METHOD_MNE] [SPACE_SURFACE] [ORIENT_FIXED] [META_END] [GRID_BEGIN] [N_SOURCES val] [N_TIMES val] [T_MIN val] [T_STEP val] [VERTEX val] ... [VERTEX val] [GRID_ROW] [AMP val] ... [AMP val] ... [GRID_END] [EOS]

Dense data layers (grid amplitudes, connectivity matrices) can be optionally omitted via InvTokenizeOptions, producing a compact metadata-only sequence suitable for context-limited attention windows.

Definition in file inv_token.h.