Detecting gestures has often been performed using non-causal techniques such as Hidden Markov Models or pick-peaking and thresholding. They can present perceptible delay that harms their use in real-time scenarios, unless a very high sampling rate is used. In this work, we investigate a lightweight CNN-based neural network to predict and anticipate musical cues (i.e., drum hits or note onsets) from accelerometer signals. We show that our architecture is able to anticipate gestures using preparatory movements, such as raising the drumstick, thus being potentially usable in music- or gaming-related interactive devices.