September 9, 2019

439 words 3 mins read

retroplasma/earth-reverse-engineering

retroplasma/earth-reverse-engineering

Reversing Google’s 3D satellite mode

repo name retroplasma/earth-reverse-engineering
repo link https://github.com/retroplasma/earth-reverse-engineering
homepage
language C
size (curr.) 1831 kB
stars (curr.) 1886
created 2019-01-12
license

Gitter Chat

header

This is an attempt to reverse-engineer undocumented parts of Google Earth. Main goal is to document the results and to provide code that emerges.

Client

here

Earth to OBJ file downloader

We can dump a textured 3D model (*.obj with *.bmp and *.jpg) using the following scripts. They require Node.js v8 and npm:

# Install dependencies (tested with node@8.15.0, npm@6.4.1)
npm install

# Find octant of latitude and longitude
node lat_long_to_octant.js 37.420806884765625 -122.08419799804688

# Dump octant with max-level 20
node dump_obj.js 20527061605273514 20

Exported files will be in ./downloaded_files/obj. They can be opened in Blender like this.

Notes

Alternative methods for finding octants:

You can use this to dump json and raw data instead of obj:

node dump_obj.js 20527061605273514 20 --dump-json --dump-raw

URL structure:

"https://kh.google.com/rt/🅐/🅑"
 - 🅐: planet
       - "earth"
       - "mars"
       - ...
 - 🅑: resource
       - "PlanetoidMetadata"
       - "BulkMetadata/pb=!1m2!1s❶!2u❷"
          - ❶: octant path
          - ❷: epoch
       - "NodeData/pb=!1m2!1s❸!2u❹!2e❺(!3u❻)!4b0"
          - ❸: octant path
          - ❹: epoch
          - ❺: texture format
          - ❻: imagery epoch (sometimes)

Misc:

General info:
 - Everything is stored in an octree.

Roles of resources:
 - PlanetoidMetadata points to first BulkMetaData.
 - BulkMetaData points to other BulkMetaData and to NodeData.
 - NodeData contains actual meshes and textures.

Versioning:
 - BulkMetaData and NodeData are versioned using epoch numbers.
 - PlanetoidMetadata provides epoch of first BulkMetaData.
 - BulkMetaData provides epochs of underlying resources.
 - Current version of a resource can be determined recursively.
 
NodeData:
 - Vertex Buffer:
    - 8 bytes per point: X,Y,Z,W,U,U,V,V
    - XYZ: position, W: octant mask, UV: texture coordinates
 - Texture:
    - Format 1: JPEG
    - Format 6: S3 DXT1 RGB
 - Raw format: see proto/README.md
BulkMetaData:
 - Oriented Bounding Box
    - Dump OBB to obj: https://gist.github.com/retroplasma/5698808bfaa63ffd03f751a84fa6ce14
    - Latlong to octant using OBB (unstable): https://github.com/retroplasma/earth-reverse-engineering/blob/443a3622ce9cb12cd4460cc6dc7999cc703ae67f/experimental_latlong_to_octant.js

TODO:
 - BVH

Related ideas: Racing game , Minimal client. WebGL + CORS should work (test).

Important

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

comments powered by Disqus