There are a lot of bosses in Kingdom Hearts 2, but you don't have to kill all of them. You should though, as each boss kill grants a bonus level, raising HP and MP.
In the original game (excluding story bosses in optional worlds) there's only one optional boss: Sephiroth. He can be challenged not long after you clear Space Paranoids, on the ledge overlooking Maleficent's heartless castle. He's a challenging fight but much easier than he was in the first game. Time your aerial recoveries to escape combos, be ready for Heartless Angel, triangle at the right times, you should be fine at higher levels. This will give you Fenrir, a strength keyblade bearing Negative Combo.
In 2.5, more bosses are present. The first you'll encounter is the absent silouette of Zexion in the Olympus Coliseum. The fight's not hard and, with some luck and some effort, he'll go down. He doesn't do much as far as attacks and is pretty easy to hit, but he will capture your allies in books, and even capture you. When in the book world you have to strike the books that attack you until you find the one controlled by Zexion. Hit it, hit the triangle prompt, and you're free, with Zexion open to some free hits. He drops a unique crafting material and the recipe for a unique magic accessory.
Marluxia is next and can be found in Beast's Castle. Only a few of his attacks damage your health because he puts a counter on you. Every scythe hit reduces the counter, which maxes out at your current level. Your best shot for getting hits in is when a whirlwind surrounds Marluxia. Block the string of attacks that follow and you'll have the choice of two reaction commands: use his scythe blade and you'll restore your count. React on Marluxia and you'll deal damage. Both of these leave him open to further attack. After some of his health is down he will pull out more attacks. Block and roll until you can get the reaction and you'll be fine. He drops the recipe for the Full Bloom, which grants 25% mp regen when upgraded to FB+
Vexen can be found in Agrabah on the second visit. You won't be able to harm him directly until you break his shield, at which point you can beat him around for a bit. Be careful though, as a circle will follow and collect data on Sora. Once it reaches a new level Vexen summons Antiform Sora. This can be annoying if it's not dealt with, and every level becomes harder. This mechanic is the real challenge of the fight. Defeating him drops the recipe for a shield.
Larxene is found on your second visit to Port Royale. There's not much of a trick to defeating her, though when she splits in two staggering both of her will allow a reaction command where you throw the two together. She drops the recipe for an armor piece, which boosts the power of thundaga in its + form.
Laexeus is last and can be found in Twilight Town. He becomes stronger as the fight progresses and some of his attacks can be hard to dodge. In order to deal damage you have to block an attack where he slams his weapon into the ground and swings it up at you, at which point he will stagger and a reaction can be unleashed to make him vulnerable. Defeating him drops the recipe for a strength-based staff.
After this your next optional bosses, called Data XIII, you can find them in the Grdfn of Assemblage, through the Cavern of Remembrance in Hollow Bastion/Radiant Garden. This consists of stronger versions of every Org XIII member, and while strategies are the same for some it takes a guide of its own. Defeating all of them will grant Proof of Nonexistence, which upgrades Sora's crown.
The final boss is the Lingering Will, found in Disney castle after completing the game. This is famously the hardest boss in the Kingdom Hearts series, so come fully leveled. It's advisable to bring Ultjma for MP regen if you heal a lot or Decisive Pumpkin for maximum damage output. Whenever you start the fight he will use an attack. Pay close attention to this and later attacks as this will tell you what moveset he's using. After a certain ammount of his health drops he switches to another moveset. This one is always the same. His attacks become more frequent and your windows to attack shrink as the fight goes on, so make surrle your timing is as early as resonable. Beating this boss grants another Proof, upgrading Sora's crown, as well as one more Drive Point.