our temp baby

We are watching my sister's little baby (6 weeks old) for a few days while she goes back to work. The kids have been slightly obsessed with him, and Hayden is a little angry whenever he's asleep and I don't let him play with him. He tells me "I hate him to be asleep". Even in the car he was a little put out that the baby was sleeping instead of playing with Hayden. I can't believe I did this before with a 2 year old and a four year old. How does it always just come together and we just manage it?