Yes, to the best of our understanding of New York laws if you are using your cell phone while in speaker mode and speaking on the phone with it held a few inches from your mouth that would be illegal.
New York state law does not allow you to use a hand-held cell phone or mobile device while you drive. In 2001, New York became the first state to prohibit drivers from talking on hand-held devices while operating a motor vehicle. You may use a hand-held cell phone or mobile device only when you call 911 or contact medical, fire or police help in an emergency. You are allowed to use your phone with a hands free device.
You can be cited by law enforcement for using a hand-held cell phone or mobile device while driving in NYS, and will be required to pay a maximum fine of $100 and mandatory surcharges of $50. If you receive a ticket for this violation the DMV does not assign any violation points to the driver's record.
Section 1225-c of the New York Vehicle and Traffic Laws is titled use of mobile telephones and notes under this part of NYS law "Using" shall mean holding a mobile telephone to, or in the immediate proximity of, the user's ear."Immediate proximity" shall mean that distance as permits the operator of a mobile telephone to hear telecommunications transmitted over such mobile telephone, but shall not require physical contact with such operator's ear.
"Hand-held mobile telephone" shall mean a mobile telephone with which a user engages in a call using at least one hand.
"Hands-free mobile telephone" means a mobile telephone that has an internal feature or function, or that is equipped with an attachment or addition, whether or not permanently part of such mobile telephone, by which a user engages in a call without the use of either hand, whether or not the use of either hand is necessary to activate, deactivate or initiate a function of such telephone.
"Engage in a call" shall mean talking into or listening on a hand-held mobile telephone, but shall not include holding a mobile telephone to activate, deactivate or initiate a function of such telephone.
In Paragraph 2 this part of the New York Vehicle and Traffic Laws it states that except as otherwise provided in this section, no person shall operate a motor vehicle upon a public highway while using a mobile telephone to engage in a call while such vehicle is in motion. An operator of a motor vehicle who holds a mobile telephone to, or in the immediate proximity of his or her ear while such vehicle is in motion is presumed to be engaging in a call within the meaning of this section.
From this information it seems clear that speaking on the phone by holding it near your mouth and ear to speak on it even if it is in speaker mode can be construed by police as you using your phone without a hands free device and thus against the law. If you would like to know more about this law you can contact the NYS Department of Motor Vehicles and read through the New York VAT laws.