Beat egg yolks, quarter cup sugar, nutmeg, salt and vanilla together with rotary beater or electric mixer until very thick and light yellow. Slowly beat in brandy and milk. Cover and chill overnight. Shortly before serving, beat egg whites to soft peaks. Gradually beat in remaining quarter cup sugar with same beater, beat cream to soft peaks. Fold meringue into beaten cream. Then slowly pour on chilled brandy mixture folding in gently.