Hi,
do you create in-process or out-process applet? out-process applets just use style info from theme directly, but in-process applets has some extra style info that is added in gnome-panel, but only for Adwaita and HighContrast themes.
How to fix it? You don't need to fix it - it is gnome-panel and/or theme problem/bug not your applet...
I have planed to rewrite gnome-panel style part, but I have no idea when I could finish it. One part of this rewrite will be that all applets will need to add panel-applet-text (right now just example) css class that will be used to set text color on panel. From panel settings user will be able to override text color that is not possible right now. When it will be done then mostly likely small changes will be needed to ensure that applet works correctly.