After gaining control of the most important port on Elba, now known as Portoferraio, Duke Cosimo I de' Medici decided to build a large fortification system to defend it. Work began on 3rd May 1548, by Architect Bellucci, later assisted by Camerini, and proceeded rapidly. In 1555, Portoferraio was the only city on the island capable of holding out against the pirate Dragut. In order to resist the Turkish pirate, in 1561 the Duke founded the Order of the Knights of Santo Stefano, which also had a base in Portoferraio, although he was never able to create a fleet capable of competing with the other seafaring powers: this was too expensive a task for the small Tuscan Duchy.
The natural structure of the bay of Portoferraio, with an isolated rocky spur embracing – and protecting - the dock, favored the defensive structure. Two fortresses were built on the higher part, on the crest of the mountain, Forte Stella and Forte Falcone; the deep, horseshoe-shaped landing place is entirely enclosed by a wall, which only opens at the Porta a Mare and is protected at one extremity by the Torre del Martello (fig. 20) or Torre della Linguella; the entrance to the water is closed by a heavy chain. From 1558 onwards, the Fronte di Terra was fortified with a powerful series of terraces which involved the collaboration of the great ducal architect Bernardo Buontalenti. Finally, the moat created in the 1600s between the ramparts and the rest of Elba transformed Portoferraio into an "impregnable island ".
The result was quite astonishing: an unassailable fortified city, which the highly accomplished British admiral, Horatio Nelson, considered to be the "safest port in the world". Finally, the Spaniards who governed Longone, now known as Porto Azzurro, built the powerful Fortezza di San Giacomo. Realized by Don Garcia de Toledo in 1603, the ramparts and star-shaped layout are reminiscent of the fortress of Antwerp. Today this fortress is a prison and only part of the building is open to visitors. On the opposite shore of the Gulf of Mola, where Porto Azzurro is situated, stands the smaller, solid Fortezza del Focardo (fig. 21), built in 1678 by Alessandro Piston for the Viceroy of Naples, Don Fernando Gioacchino Foscardo. Unlike Portoferraio, this town became a lively point of reference for Neapolitan, Spanish, English and French pirates against attacks by the Muslim fleet.