auto-rig

A character nobody rigged, walking. Four meshes as they came out of a modelling program (no skeleton, no weights) are rigged in the browser and play UAL's clips: eight markers per mesh place a 65-bone skeleton (fitSkeleton), bone heat binds the skin to it (boneHeat), buildRig makes a Babylon skeleton of the result, and retargetClip plays the clips on it. The weasels have tails, which no clip animates, so their tails run on spring bones.

Demo

Omnidude, a weasel, a skinny weasel and a space marine (shown at 42% of his 4.4 m), by Tonio Loewald, CC BY 4.0.

Open the panel (the owl) to rig one yourself: pick a character and its front view appears with the eight markers on it. Drag a marker (the other side follows while symmetry is on) and when you let go the character is fitted, bound and walking again. guess places the markers from the silhouette alone, which is where you would start with a new mesh. The weasels' tail tips are placed in the side view. The clip, the bones and tail stiffness are below.

import {
  b3d, b3dLight, assetUrl, select3d, slider3d, toggle3d, button3d, label3d,
  objTriangles, fitSkeleton, meshQuery, skinSegments, boneHeat, buildRig,
  readSkeleton, retargetClip, springNodes, markerField3d,
} from 'tosijs-3d'
import { tosi } from 'tosijs'

const { autorig } = tosi({
  autorig: { who: 'skinny weasel', clip: 'Walk_Loop', bones: false, tail: 0.25, symmetry: true },
})
const clips = ['Idle_Loop', 'Walk_Loop', 'Jog_Fwd_Loop', 'Sprint_Loop', 'Dance_Loop', 'Crouch_Fwd_Loop']

// Mixamo's eight markers on the front view (the character's left is +X),
// given for the left side and mirrored.
const markers = (chin, groin, wrist, elbow, knee) => {
  const p = ([x, y]) => ({ x, y })
  const m = ([x, y]) => ({ x: -x, y })
  return {
    chin: p(chin), groin: p(groin),
    wristL: p(wrist), wristR: m(wrist), elbowL: p(elbow), elbowR: m(elbow), kneeL: p(knee), kneeR: m(knee),
  }
}
const cast = [
  { name: 'omnidude', file: 'omnidude.obj', objects: ['Omnidude', 'Head'], color: [0.55, 0.6, 0.7],
    markers: markers([0, 1.338], [0, 0.467], [0.74, 1.075], [0.47, 1.095], [0.152, 0.258]) },
  { name: 'weasel', file: 'weasel.obj', objects: ['Weasel'], color: [0.6, 0.42, 0.25], tail: { x: 0, y: 0.5, z: -0.95 },
    markers: markers([0, 0.967], [0, 0.358], [0.51, 0.763], [0.397, 0.763], [0.142, 0.174]) },
  { name: 'skinny weasel', file: 'weasel-skinny.obj', objects: ['Body', 'Head', 'Tail'], color: [0.8, 0.5, 0.25], tail: { x: 0, y: 0.77, z: -0.95 },
    markers: markers([0, 1.259], [0, 0.64], [0.657, 1.154], [0.404, 1.14], [0.106, 0.34]) },
  { name: 'space marine', file: 'space-marine.glb', mesh: 'EvilMarineBody', color: [0.3, 0.4, 0.3], scale: 0.42,
    markers: markers([0, 3.594], [0, 1.656], [1.966, 2.937], [1.18, 3.064], [0.388, 0.943]) },
]

let bodies = []
let field = null

// Tall enough for the marker panel beside the characters.
preview.closest('tosi-example')?.style.setProperty('--tosi-example-height', '560px')
preview.append(
  b3d(
    {
      style: 'width:100%;height:100%',
      clearColor: '#2a3340',
      panelTitle: 'Auto-rig',
      scenePanel: () => {
        const i = cast.findIndex((c) => c.name === autorig.who.value)
        const body = bodies[i]
        field = body
          ? markerField3d({
              positions: body.positions, indices: body.indices,
              value: body.markers, tail: body.tail, symmetry: autorig.symmetry.value,
              handleCommit: (m, t) => { body.markers = m; body.tail = t; body.rig() },
            })
          : null
        return [
          select3d({ label: 'rig', value: autorig.who, options: cast.map((c) => c.name) }),
          field ?? label3d({ text: 'loading…' }),
          toggle3d({ label: 'symmetry', value: autorig.symmetry }),
          button3d({ label: 'guess', handleClick: () => field?.guess() }),
          select3d({ label: 'clip', value: autorig.clip, options: clips }),
          toggle3d({ label: 'bones', value: autorig.bones }),
          slider3d({ label: 'tail stiffness', value: autorig.tail, min: 0, max: 1, step: 0.01 }),
        ]
      },
      async sceneCreated(el, BABYLON) {
        const scene = el.scene
        const cam = new BABYLON.ArcRotateCamera('c', Math.PI / 2, 1.35, 6.5, new BABYLON.Vector3(0, 0.8, 0), scene)
        cam.attachControl(el.querySelector('canvas'), true)
        el.setActiveCamera(cam)
        const ground = BABYLON.MeshBuilder.CreateGround('g', { width: 12, height: 6 }, scene)
        ground.position.y = -0.01

        // The meshes, as triangles in the file's own (right-handed) frame.
        const triangles = async (c) => {
          const url = assetUrl('tonio/rigging/' + c.file)
          if (c.file.endsWith('.obj')) return objTriangles(await (await fetch(url)).text(), c.objects)
          const glb = await BABYLON.LoadAssetContainerAsync(url, scene)
          const m = glb.meshes.find((m) => m.name === c.mesh)
          const out = { positions: Array.from(m.getVerticesData('position')), indices: Array.from(m.getIndices()) }
          glb.dispose()
          return out
        }

        // UAL's mannequin is only the clips' source: it stays hidden.
        const ual = await BABYLON.LoadAssetContainerAsync(assetUrl('quaternius/UAL1_core.glb', 2), scene)
        ual.addAllToScene()
        for (const g of ual.animationGroups) g.stop()
        const ualRoot = ual.meshes.find((m) => m.name === '__root__')
        const source = readSkeleton(ual.skeletons[0], ualRoot)
        ualRoot.setEnabled(false)

        // Retarget a clip to a body the first time it is asked for.
        const clipFor = (body, name) => {
          if (!body.clips.has(name)) {
            const clip = ual.animationGroups.find((g) => g.name === name)
            body.clips.set(name, clip ? retargetClip(clip, source, body.read, { scene }) : null)
          }
          return body.clips.get(name)
        }
        const play = (body) => {
          body.playing?.stop()
          body.playing = clipFor(body, autorig.clip.value)
          body.playing?.start(true)
        }

        bodies = []
        for (const [i, c] of cast.entries()) {
          const { positions, indices } = await triangles(c)
          const mat = new BABYLON.StandardMaterial(c.file, scene)
          mat.diffuseColor = new BABYLON.Color3(...c.color)
          // Place and scale by a node above the rig's own (mirrored) root.
          const stand = new BABYLON.TransformNode(c.file + '-stand', scene)
          stand.position.x = 2.4 - 1.6 * i
          stand.scaling.setAll(c.scale ?? 1)
          const body = { positions, indices, markers: c.markers, tail: c.tail ?? null, clips: new Map() }
          // Fit, bind and build, then play: again whenever the markers move.
          body.rig = () => {
            for (const g of body.clips.values()) g?.dispose()
            body.clips.clear()
            body.playing = null
            body.spring?.dispose()
            body.viewer?.dispose()
            body.built?.dispose()
            const rig = fitSkeleton(body.markers, meshQuery(positions, indices), { tail: body.tail ?? undefined })
            const weights = boneHeat(positions, indices, skinSegments(rig))
            const built = buildRig(scene, c.file, positions, indices, rig, weights)
            built.mesh.material = mat
            built.root.parent = stand
            const tail = ['tail_01', 'tail_02', 'tail_03', 'tail_04', 'tail_05', 'tail_leaf'].map((n) => built.node(n))
            body.spring = tail[0] ? springNodes(scene, tail, { params: { stiffness: autorig.tail.value } }) : null
            body.viewer = new BABYLON.SkeletonViewer(built.skeleton, built.mesh, scene, false, 3, { displayMode: 0 })
            body.viewer.isEnabled = autorig.bones.value
            body.built = built
            body.read = readSkeleton(built.skeleton, built.root)
            play(body)
          }
          body.rig()
          bodies.push(body)
        }
        el.refreshScenePanel()

        autorig.who.observe(() => el.refreshScenePanel())
        autorig.symmetry.observe(() => { if (field) field.symmetry = autorig.symmetry.value })
        autorig.clip.observe(() => { for (const b of bodies) play(b) })
        autorig.bones.observe(() => { for (const b of bodies) b.viewer.isEnabled = autorig.bones.value })
        autorig.tail.observe(() => { for (const b of bodies) if (b.spring) b.spring.params.stiffness = autorig.tail.value })
      },
    },
    b3dLight({ intensity: 0.9 })
  )
)

How it is built

AUTO-RIG, THE BABYLON SIDE: a fitted skeleton (rig-fit) and its skin weights (bone-heat) become a real Babylon skeleton on a mesh, built the way the glTF loader builds one, so everything that works on a loaded character works on it: readSkeleton and retargetClip play UAL clips on it, and springNodes simulates its tail.

The joints are TransformNodes under one container (root), each bone LINKED to its node, with the mesh under the same container: Babylon then reads each bone from its node, and the skin and the nodes share a frame.

THE DATA IS RIGHT-HANDED, as OBJ and glTF files are (and as rig-fit assumes: a character facing +Z has its left on +X). Babylon is left-handed, so the container carries the same mirror glTF's __root__ does (a half turn about Y and z scaled by -1). Pass root to readSkeleton as you would __root__: both measure in the file's own frame. Move the character by a parent of root, not root itself.