The first stamp to be produced was by Great Britain in 1840. This page provides links to each of 1st issues by each Country.
Great Britain: The Penny Black
South_Africa_1910_Opening_of_Parliament