Joss Whedon says he’s "torn" about directing an Avengers sequel, according to the Los Angeles Times.  His reservations have to do with the amount of work "telling what is ultimately somebody else’s story, even though I feel like I did get to put myself into it."  Whedon did tell the LA Times that he had "a bunch of ideas" for the sequel, and that "they all seem really cool."

Whedon may not have to decide for a while; Marvel has thre movies that have to come out before an Avengers sequel (Iron Man, Thor and Captain America sequels, see "Audiences Will Have to Wait for 'Avengers 2'").