The Earth's shape, size and datum
Navigation starts with a model of the Earth that is simple enough for calculation and accurate enough for positions from charts, radio aids and satellite systems to agree.
The shape of the Earth
For most navigation calculations the Earth is treated as a sphere. Its real shape is an oblate spheroid, slightly flattened at the poles and wider at the equator. Rotation during formation produced the equatorial bulge. The sources use the rounded working pair of approximately 1 in 300 and 0.3 per cent for the flattening, also called compression. Bali gives the more exact geodetic ratio as about 1 in 297.
Axes, radii and compression
The equatorial diameter is about 43 km, or roughly 23 NM, greater than the polar diameter. Bali lists an equatorial radius of 6 378 137 m, rounded in his table to 3443 NM, and a polar radius of 6 356 752 m, about 3432 NM. Direct conversion of the equatorial value gives 3443.9 NM. The difference is small compared with the full size of the Earth, so a sphere is an excellent working model for routine flight calculations.
Worked semi-minor axis
If the semi-major axis is 6378.4 km and the supplied compression is 1/297, the reduction is 6378.4 divided by 297, which is 21.48 km. The semi-minor axis is therefore 6378.4 minus 21.48, giving about 6356.9 km.
Geoid and geodesy
The irregular, gravity-related figure that best represents mean sea level continued beneath the continents is the geoid. The word means Earth-shaped. Geodesy is the science of measuring and modelling the Earth's size, shape, gravity field and positions. A mathematical ellipsoid is smooth and convenient; a geoid follows the irregular gravity surface. A geodetic datum fixes an ellipsoid's size, orientation and relationship to the Earth.
WGS-84
Different local datums can assign slightly different coordinates to the same physical point. That mattered little when navigation accuracy was coarse, but it matters to satellite navigation, inertial systems, flight management systems and stored radio-aid coordinates. ICAO uses the World Geodetic System 1984, WGS-84, as the global reference for international air navigation. Charts, databases and position sensors must use a common datum to avoid a position shift.
“The Earth's shape is commonly described as an oblate spheroid.”Oxford ATPL Book 10, chapter 1
“Earth is an oblate compressed spheroid or ellipsoid.”R.K. Bali, Air Navigation, chapter 2
The poles and rotation
The geographic North and South Poles are the ends of the axis about which the Earth spins. The spin is from west to east. Viewed from above the North Pole it appears anticlockwise; viewed from above the South Pole it appears clockwise. The polar axis is inclined by about 23.5 degrees to the normal to the plane of the Earth's orbit. That inclination becomes important when time, seasons and daylight are studied.
Direction on the Earth
A direction is an angle measured clockwise from a stated north reference. The angle is incomplete unless the reference is identified.
Cardinal and quadrantal points
The cardinal points are North, East, South and West. The four quadrantal points lie midway between them: north-east, south-east, south-west and north-west. These names are useful for broad orientation, but precise air navigation uses degrees.
| Point | Three-figure direction | Opposite direction |
|---|---|---|
| North | 000 degrees or 360 degrees | South, 180 degrees |
| North-east | 045 degrees | South-west, 225 degrees |
| East | 090 degrees | West, 270 degrees |
| South-east | 135 degrees | North-west, 315 degrees |
| South | 180 degrees | North, 000 degrees |
| South-west | 225 degrees | North-east, 045 degrees |
| West | 270 degrees | East, 090 degrees |
| North-west | 315 degrees | South-east, 135 degrees |
The sexagesimal system
A complete circle contains 360 degrees. Direction is measured clockwise from north through east, south and west. North is 000 degrees, east 090 degrees, south 180 degrees and west 270 degrees. Three figures are used, so write 005 degrees and 090 degrees, not 5 degrees and 90 degrees. When geographic north is the reference, attach the suffix T for true.
A reciprocal direction is 180 degrees away. Add 180 when the original is below 180; subtract 180 when it is 180 or more. The reciprocal of 060 degrees is 240 degrees. The reciprocal of 353 degrees is 173 degrees.
Runway numbers are a special shorthand
A runway designator represents its magnetic direction rounded to the nearest ten degrees, with the final zero omitted. A direction near 273 degrees becomes runway 27; a direction near 078 degrees becomes runway 08. The reciprocal runway number differs by 18, allowing for the 01 to 36 range.
True, magnetic and compass direction
True north belongs to the graticule, magnetic north belongs to the Earth's magnetic field, and compass north is what the aircraft compass actually indicates.
The three north references
| Reference | Meaning | Suffix |
|---|---|---|
| True north | Direction along a meridian towards the geographic North Pole. | T |
| Magnetic north | Direction of the horizontal component of the Earth's magnetic field towards the magnetic north pole. | M |
| Compass north | Direction indicated by the aircraft magnetic compass after local aircraft magnetic effects act on it. | C |
Variation or magnetic declination
Variation is the angular difference between true north and magnetic north at a place. If magnetic north lies east of true north, variation is easterly and positive. If it lies west, variation is westerly and negative. Variation changes with position and slowly with time, so it is taken from a current chart. An isogonal joins places of equal magnetic variation. The zero-variation line is an agonic line.
Deviation
Deviation is the angular difference between magnetic north and compass north. It is caused by magnetic fields within the aircraft. It changes with heading and can also change after equipment, structure or electrical systems are altered. The value for a heading is taken from the current compass deviation card. Easterly deviation is positive; westerly deviation is negative.
The C D M V T chain
Write the references in order: Compass, Deviation, Magnetic, Variation, True. When working from compass towards true, add signed deviation and signed variation. East is positive and west is negative.
Worked example from compass to true
Compass heading 086 degrees, deviation 7 degrees east and variation 3 degrees west. Magnetic heading is 086 plus 7, giving 093 degrees M. True heading is 093 minus 3, giving 090 degrees T.
Worked example from true to compass
True heading 270 degrees, variation 4 degrees east and deviation 6 degrees west. Magnetic heading is 270 minus 4, giving 266 degrees M. Compass heading is 266 minus negative 6, giving 272 degrees C.
Circles, meridians and the graticule
Latitude and longitude replace flat Cartesian coordinates with two angular coordinates built from circles on the Earth's surface.
Great circles and small circles
A great circle has the same centre and radius as the Earth, so its plane divides the Earth into two equal hemispheres. The shorter great-circle arc between two points is the shortest surface distance. Only one great circle passes through two points unless they are diametrically opposite, in which case infinitely many do.
A small circle is any surface circle whose centre and radius are not those of the Earth. Every parallel of latitude except the equator is a small circle. A rhumb line crosses every meridian at the same angle and therefore maintains a constant direction. Except for a meridian or the equator, it is not the shortest route and spirals towards a pole if continued. Great circles and rhumb lines are treated fully in Chapter 2.
| Line | Geometrical character | Navigation property |
|---|---|---|
| Great circle | Plane passes through Earth's centre. | Shorter arc is the shortest surface route. |
| Small circle | Plane does not pass through Earth's centre. | Includes all parallels except the equator. |
| Rhumb line | Cuts all meridians at one constant angle. | Maintains constant direction but is normally longer than the great circle. |
The equator
The equator is the great circle whose plane is perpendicular to the spin axis. It divides the Earth into Northern and Southern Hemispheres, runs east and west, and is the datum for latitude. It is both a great circle and a rhumb line.
Meridians and the prime meridian
A meridian is a semi-great circle joining the geographic poles. It joins points of equal longitude, crosses the equator at right angles and indicates true north and south. A meridian together with its anti-meridian forms a complete great circle. The meridian through Greenwich is the prime meridian, the zero datum for longitude. Its anti-meridian is the coincident 180 degrees east and 180 degrees west line.
Parallels of latitude
A parallel joins places of equal latitude. Its plane is parallel to the equator, it runs east and west and crosses every meridian at right angles. Parallels remain parallel to one another. Their radii and circumferences shrink towards the poles.
The graticule
The network of meridians and parallels on a globe, map or chart is the graticule. Meridians converge at the poles and are farthest apart at the equator. Parallels never converge. Their intersections create an unambiguous angular position reference system.
“The network formed by the meridians, equator and parallels is the graticule.”Oxford ATPL Book 10, chapter 1
“Graticule is the network of meridians and parallels on a map or chart.”R.K. Bali, Air Navigation, chapter 2
Latitude and change of latitude
Latitude tells how far north or south a point lies, as an angle from the equatorial plane rather than as a direct linear distance.
Angular units
A circle contains 360 degrees. One degree contains 60 minutes of arc, and one minute contains 60 seconds of arc. The arc-minute symbol is ', while the arc-second symbol is ". Decimal minutes may be used instead of seconds, but the two formats must not be mixed carelessly.
| Angular quantity | Equivalent |
|---|---|
| 1 degree | 60 minutes of arc |
| 1 minute of arc | 60 seconds of arc |
| 1 degree | 3600 seconds of arc |
Latitude defined
Geocentric latitude is the smaller angle between the equatorial plane and the line joining the Earth's centre to the point. It is written from 00 degrees at the equator to 90 degrees north or south at the poles. Latitude is always stated before longitude in a coordinate.
Geocentric and geodetic latitude
Because the Earth is oblate, the normal to the ellipsoid at a point does not usually pass through the Earth's centre. Geodetic latitude is the smaller angle between that normal and the equatorial plane. Navigation charts use geodetic latitude. The maximum difference between geocentric and geodetic latitude occurs near 45 degrees north or south and is about 11.6 minutes of arc. At the equator and poles the two definitions agree.
Special parallels
| Parallel | Latitude | Reason it is named |
|---|---|---|
| Tropic of Cancer | 23.5 degrees N | Northern limit of the Sun's overhead position. |
| Tropic of Capricorn | 23.5 degrees S | Southern limit of the Sun's overhead position. |
| Arctic Circle | 66.5 degrees N | Polar daylight and darkness boundary associated with axial tilt. |
| Antarctic Circle | 66.5 degrees S | Southern equivalent of the Arctic Circle. |
Difference or change of latitude
Difference of latitude, written d'lat or change of latitude, is the smaller angular difference between two latitudes. If both points are in the same hemisphere, subtract the smaller latitude from the larger. If they are in opposite hemispheres, add their latitude values. The result is north or south according to the direction from the first point to the second.
Worked examples
From 40 degrees N to 10 degrees N, d'lat is 30 degrees south. From 40 degrees N to 30 degrees S, d'lat is 70 degrees south. From 20 degrees 30 minutes N to 41 degrees 30 minutes S, the contrary names are added, giving 62 degrees south.
Longitude and change of longitude
Longitude identifies the meridian through a point by measuring the shorter angle east or west from Greenwich along the equator.
Longitude defined
Longitude is the smaller angle at the Earth's axis, or equivalently the shorter equatorial arc, between the prime meridian and the meridian through the point. Values run from 000 degrees to 180 degrees east or west. The 180 degrees east and 180 degrees west meridians coincide as the Greenwich anti-meridian.
Difference or change of longitude
Difference of longitude, written d'long or change of longitude, is the smaller angular difference between two meridians. For longitudes with the same name, subtract. For unlike names, add. If that sum exceeds 180 degrees, subtract it from 360 degrees to obtain the shorter difference. State the direction east or west from the first position to the second.
Worked examples
- From 165 degrees W to 103 degrees W, subtract to obtain 62 degrees. The destination lies east of the start, so the change is 62 degrees E.
- From 30 degrees W to 30 degrees E, add to obtain 60 degrees E.
- From 163 degrees E to 152 degrees W, the first sum is 315 degrees. The shorter difference is 360 minus 315, giving 45 degrees E.
The apparent reversal near 180 degrees
East as a direction always remains 090 degrees T and west remains 270 degrees T. Near Greenwich, eastern longitudes lie physically east and western longitudes lie physically west. Near the anti-meridian, an eastern longitude can lie to the west of an observer while a western longitude lies to the east. Keep the direction of travel separate from the E or W name attached to the coordinate.
Latitude and longitude are different systems
| Feature | Latitude | Longitude |
|---|---|---|
| Datum | Equator | Prime meridian |
| Range | 0 to 90 degrees N or S | 0 to 180 degrees E or W |
| Reference line through a point | Parallel of latitude | Meridian |
| Behaviour | Parallels never meet | Meridians converge at both poles |
| Distance for 1 minute | About 1 NM along a meridian | 1 NM only along the equator, decreasing towards the poles |
Positions, distance, resolution and vertices
A coordinate states latitude first and longitude second. Its number of digits declares the resolution, while great-circle geometry relates the angular values to distance and vertices.
Writing a position
Write latitude first, then longitude. Latitude uses two degree digits; longitude uses three. Each must carry its hemisphere letter. A position may be written in degrees and whole minutes, degrees and decimal minutes, or degrees, minutes and seconds.
| Format | Example | Implied latitude resolution | Typical use |
|---|---|---|---|
| Whole minutes | 2844N 07706E | 1 NM, about 6080 ft | En-route plotting |
| Decimal minutes | 2844.3N 07706.5E | 0.1 NM, about 608 ft or 185 m | Navigation-system entry and display |
| Whole seconds | 28°44'18"N 077°06'30"E | About 101 ft or 30 m | Aerodrome and detailed chart data |
| Tenths of a second | 28°44'18.3"N | About 10 ft or 3 m | Precision survey information |
| Hundredths of a second | 28°44'18.32"N | About 1 ft or 30 cm | High-resolution calibration data |
Angular arc and distance
The ICAO nautical mile is exactly 1852 m. One minute of arc on a great circle is treated as one nautical mile for navigation. Bali gives the exact conversion as about 6076 ft and Oxford uses the traditional working value of about 6080 ft. The spherical circumference is 360 times 60, giving 21 600 NM.
The same direct conversion applies to change of longitude only on the equator, because only there is a parallel also a great circle. Away from the equator, meridians are closer together and one minute of longitude represents less than one nautical mile.
Resolution is not guaranteed accuracy
Writing more digits expresses a finer coordinate resolution. It does not prove that the survey, chart, sensor or database is accurate to that level. A position written to 0.1 minute identifies increments of about 185 m along a meridian; a receiver might be more or less accurate depending on its integrity and operating conditions.
Great-circle vertices
The northern vertex is the most northerly point of a great circle and the southern vertex the most southerly. The two vertices are antipodal, lie on opposite meridians, have equal latitude values with opposite names, and are 10 800 NM apart along the great circle. At a vertex the great-circle direction is 090 degrees T or 270 degrees T.
Each equator crossing is 90 degrees of longitude from a vertex. If the southern vertex is 63 degrees S 170 degrees W, the northern vertex is 63 degrees N 010 degrees E. The great circle crosses the equator at 080 degrees W and 100 degrees E. Tracking east from the northern side, the two equator-crossing tracks in Oxford's example are 153 degrees T and 027 degrees T. The reciprocal westbound values are 333 degrees T and 207 degrees T.
| Vertex fact | Rule |
|---|---|
| Opposite vertex | Reverse the latitude name and change longitude by 180 degrees. |
| Distance between vertices | Half the Earth's working circumference, 10 800 NM. |
| Direction at a vertex | 090 degrees T or 270 degrees T. |
| Equator crossing | 90 degrees of longitude from either vertex. |
| Meridian special case | Vertices at the poles; it crosses the equator northbound or southbound. |
| Equator special case | Vertices lie on the equator; direction is east or west throughout. |