Cut the French bread into 1 inch pieces and place in 9x13 baking dish lightly coated with cooking spray.
In a large bowl, combine eggs, half and half, milk, pumpkin puree, pumpkin pie spice, brown sugar and salt. Whisk to combine.
Pour over bread pieces and gently mix to ensure all of the bread is coated with egg mixture. Cover the dish and refrigerate overnight.
Remove French toast from refrigerator and uncover. Preheat oven to 350 degrees.
Combine the brown sugar, white sugar, pumpkin pie spice and nutmeg for the topping in a medium bowl and stir well to combine. Cut in butter pieces to create a crumbly topping. Sprinkle topping over bread mixture.
Bake in preheated oven until top is golden brown and middle is set; about 50 minutes.