I think you can make the case of Sam being the one who was always outside looking in -- and worked at being in as much as he could, whereas Dean may have looked in but decid3ed it was not for them.