With drywall dust, a shop vac is going to become your best friend. Its open chamber is perfect for sucking up the tiny ...

Use a vacuum, not a brush ... and they have higher filtration to keep out drywall dust particles. You can get them with a pair of goggles to keep the dust out of your eyes for about $20.