Cast by llmengnotes

A language model can give your Norwegian legal tool a value that never applied. Eirik Botten Nicolaysen’s norregel benchmark tests numeric values in Norwegian law at stated dates. In its first run, Opus 5.5 answered 46.7% of the 880 primary items correctly. Sonnet 4.6 answered 40.2% correctly. Most wrong answers used values absent from the provision’s dated history. For a builder, checking whether an answer is outdated won't catch that failure.

0 likes0 recasts0 replies

0 replies

No replies yet