Firstly if you want to read a comprehensive article about jpgs and gifs then go to Wikipedia. That's what i've read to understand why the two formats exist, actually the gif article is great but you'll need a maths degree to decifer the jpg article.
To summarize, they both take an image and compress it / make the file size smaller. To compress an image in this way something has to be thrown away. In jpgs this is picture information, in gifs this is colours.
I've found no simple article explaining what a jpg does to compress, probably because it's hugely complicated. So in my (idiot's) terms, 'A jpg will throw away information by grouping together pixels. This is what gives you that blurry edge on a heavily compressed jpg image. However because it can contain millions of colours, photos are still as vivid after compression.
A gif can only use 256 colours, therefore if you have turn a photo into a gif then it becomes grainy as some of the colour information has to be thrown away. However because gifs don't throw picture information away all sharp edges stay sharp.
Used in the right way you can have images online that are compressed but don't LOOK compressed. The following articles from the grammatically incorrect Web Developers Notes (Web Developer's Notes or even Web Developers' Notes!) contain good images of how the compression works and why some images need to be gif and some jpgs (the pages are fit to burst with annoying advertising but then i'm not suggesting this site is accessible).
When to use gifs - "Use gifs for line art, cartoons and images that contain text because they preserve the sharp lines of shapes and text in the graphic" ...and also animated images (as you can't animate jpgs).
When to use jpgs - "Use jpgs for photographs or other high colour images because they preserve gradients and subtle colours."
I've just found a minutely less complicated article about how a jpg compresses things, i got to the second paragraph, can you beat me?