Notochord

The notochord is a flexible, rod-shaped body found in embryos of all chordates. It forms the axis of the embryo. In some chordates, it persists throughout life as the main support down the length of the body. In most vertebrates it is replaced by the vertebral column. The notochord is found ventral (underneath) the neural tube (future spinal cord). 
Notochords were the first "backbones" in chordates. Notochords were advantageous to primitive fish-ancestors: they were a rigid structure for muscle attachment, but flexible enough to allow movement. Embryos of vertebrates have notochords, and develop the vertebral column later.