It's kind of like a drug, the kind of drug that any martial artist is familiar with. You start to figure one system out, and, just as it starts to feel like you're doing the same old thing again and again, you notice a shiny new thing to check out and, hey! This new thing is pretty close to the old thing, but because I've got a bit of experience I can pick it up easier. Hey, and it expresses some new ideas that make a lot of sense! They make more sense than the first one did, that's for sure. Oh, but wait, there's another one over here...
At the school I teach at there are a number of different classes you can take, most of them based in the same theory, but with idiosyncratic differences from weapon to weapon: rapier, longsword (both on foot and mounted), polearms (on foot and mounted), and sidesword, not to mention grappling (also on foot and mounted). Outside of the core, single-system weapons we've also got Bartitsu (which itself consists of different disciplines, to some extent), stage combat, and archery (on foot and mounted). That's, what, up to a dozen different things to work on? Fifteen, if you count the components of Bartitsu separate things. It's a veritable feast of choice, but I highly recommend against doing everything at once, unless you want to have a really frustrating experience.
The biggest issue is time, and skill acquisition. It's hard to learn new things, and once a week, to be frank, doesn't cut it for most people. Generally, once a week is just long enough for you body to have forgotten what it learned the previous week. For someone who's relatively new, twice a week is pretty good for starting out, but to really get things down you should really be working on it three or four times a week. And, yes, those should be sessions with an instructor, for at least an hour. It's good to practice the basics on your own, but it's not good to practice the wrong thing, or the right thing in the wrong way. When you're starting out your practice time should be spent learning the viscera of whatever you're doing: get the terminology down, and classes become much easier. That means 1-2 hours of class time, 2-4 times per week. If you're splitting that time up between all of the above (rapier, longsword, and bartitsu, say) you just aren't going to learn them nearly as fast as if you dedicated the same amount of time to each of them in turn.
The good news is that once you've got the basic skill set down pat, it's relatively easy to maintain that skill. Get yourself up to an intermediate/advanced level with something, and weekly sessions are often enough to keep the better part of the rust off. Let too much time elapse without training, however, and you'll be amazed at how much your limbs have forgotten how to do. This means that diversifying is okay, as long as you're building a solid base in each area before moving on to other things. And the closer the theory of each of the systems are to each other, the easier it is to learn and transfer skill between them: Learning Fiore's dagger, longsword, and wrestling works together, because they all share commonalities and build on each other.
There's so much information out there, and it's very tempting to learn everything. But, and this is the hardest part for me, it's also so easy to drop something once it starts getting difficult. Sticking to something for the long haul is sometimes the only way to really find out what it has to offer.
Final anecdote: I see a lot of guys doing amazingly well at the HEMA thing, some of whom are out and teaching classes and being hailed as experts with as few as two years of study of the subject. The other day I had a kendo guy come in to the store, and we were talking a bit about swordplay. When I asked him what he thought of it he said, "Well, I'm just getting started; I've only been doing it for five years."