Development in my 2 year old

I have an issue where my 2 year old daughter is doing addition.  She does not speak many words, but she seems to have a way with numbers.  Her mechanical skills are also great.  She knows all of her colors, knows all of her letters and can even identify animals that start with the corresponding letter.  If I show her a picture of a cat she will point to the letter c or if I tell her banana she will pick up a letter b.  

Our pediatrician was concerned that her language skills aren't great.  The state of Pennsylvania sent over two women to evaluate her.  They both said she was above average in everything but vocabulary.  They decided she didn't need a speech pathologist just a teacher.  Now we have a woman coming over every other week and she basically just plays with her.  She insists that we shouldn't be doing the math with her, but she loves doing it.  She tells us we should be working om really simple stuff.

My problem is that my daughter will get extremely bored if we do really basic stuff.  She is very in tune to the numbers and actually tries to say them.  Her vocabulary is actually starting to get better , and I want to continue with the math because she likes doing it.  The woman from the state wants her to sign the words to us.  I just feel like what we are doing is starting to work, and don't want to backtrack because someone from the state says to.  What would you parents do???  Please please share your opinion