Preheat the oven to 350 degrees. Prepare a 9x13 casserole dish with nonstick cooking spray.
Pour the rice into the casserole dish.
Place the celery and onion into a microwave safe dish. Cover the dish with plastic wrap or waxed paper. Microwave on high for 2 minutes or until the vegetables have softened. Add the softened vegetables over the rice.
Over the vegetables, add the seasonings.
In a bowl, whisk together the cream of chicken soup, chicken broth, and water. Pour the mixture over the seasonings and stir the rice to combine.
Place the chicken over the casserole. Brush the chicken drumsticks with the melted butter and sprinkle with salt, black pepper, and the remaining paprika.
Cover the casserole tightly with aluminum foil. Bake for 1 hour.
Carefully remove the aluminum foil. Stir the rice around the chicken. (You can also remove the chicken drumsticks to stir the casserole but return them before continuing to bake.)
Place the casserole back into the oven, uncovered, and bake for an additional 30 minutes or until the chicken reaches an internal temperature of 165 degrees.