A lot of people who should know better are referring to the year 2000 as the start of the next millennium. This is careless use of terminology.
A millennium is a period of 1,000 years. Every year is included in 1,000 different millennia. For example, 1999 is the first year of a 1999-2998 millennium, the second year of a 1998-2997 millennium, all the way to the last (1,000th) year of a 1000-1999 millennium. If someone says 1999 is the last year of “a” millennium or 2000 is the start of “a” millennium, they are correct. If someone says 1999 is the last year of “the” millennium or 2000 is the start of “the” millennium, they are probably incorrect. If they introduce the qualifier “the millennium that started in the year 1000”, it is correct to say 1999 is the end of that millennium. If they are speaking in terms of the calendar used by most of the world, they are incorrect.
The current calendar, in which the current year is 1999, goes directly from 1 B.C. (before Christ) to 1 A.D. (Anno Domini, in the year of our Lord) or 1 C.E. (common era) for the non-religious version. There is no year zero in this system, therefore the first millennium C.E. was the years 1 through 1000, the second millennium C.E. is the years 1001 through 2000, and the third millennium C.E. will be the years 2001 through 3000. The year 2000 is the last year of the second millennium C.E., not the first year of the third. Also, it is almost certain that the B.C./A.D. boundary was set wrong when the Christian Era was invented by Dionysius Exiguus in 525 A.D. and 1 C.E. is several years off from the birth of Christ, so the real Christian second millennium probably ended a few years ago.
The lack of a year zero in B.C./A.D. (C.E.) dates is understandable, and makes a good example of basic psycholinguistics. B.C./A.D. dates have no year zero because the creators literally could not represent the idea. When the Christian Era dating system was first established in Italy, Roman Numerals (which have no symbol for zero) were standard notation. Arabic Numerals, with zero, came to Europe centuries later.
Centuries follow rules similar to the rules for millennia. Every year is a member of 100 centuries, but the centuries of C.E. dates started with the first century made up of the years 1 through 100. The 20th century C.E. is the years 1901 through 2000. The 21st century C.E. will be the years 2001 through 2100.
However, with centuries, there is another common usage. The year 1999 is the last year of the 1900s. The year 2000 will be the first year of the 2000s. This usage describes centuries which share their first digits and starts and ends one year earlier than the nearest matching C.E. century.
The (in)famous Y2K (year 2000) bug does have to do with the end of the 1900s and the beginning of the 2000s, not with the end of the 20th century C.E. or the 2nd millennium C.E. The “bug” relates to using only the last two digits of years, and is triggered by the change of the common-digits century, not the C.E. century.
Decades are usually named based on common digits (1980s, 1990s, 2000s, etc.), so they are in synch with common-digits centuries, not C.E. centuries or millennia. The 1900s decade was the first decade of the 1900s century, and the 1990s are the last.