This happened to the Jets app that BoltOps is running. Actually run quite a few of them now. Looks like AWS is rolled out the update more generally on 4/16/2020. Thanks to this report knew exactly what to do! Upgrade json 2.3. Bummer again about it!
Also, this PR should migrate the issue somewhat https://github.com/tongueroo/jets/pull/470 since it removes json from Gemfile.lock. Though projects or other gems can still add json, it at least migrates it.