Part of this ideological structure is the institution of animal welfarism which teaches people that animal use is not the issue; animal treatment is, especially in relation to nonhuman individuals held as pets.
Even though many people may say that they "do not care" about other animals, they rarely think we are free to do absolutely anything we want to do to them. It is not often, for example, that we'd meet a person who sees nothing wrong when they hear news that someone put a small animal into a microwave oven, or has slashed horses in fields, or likes to have sex with nonhuman animals until they die. People may suggest that their opposition to these things is based on indirect duties we have to other animals but many will recognise that we have some direct duties to them that would not allow such forms of use.
The reality is that they'd prefer to keep the nuts and bolts of the way we use animals out of their consciousness. The very existence of vegans (and even vegetarians) reminds them of things they'd rather forget; things they go out of their way to "not know."
We should recognise that we are still in the pioneer stage when it comes to veganism.
The term only came into being in 1944 and veganism as the moral baseline of the movement is very new as an accepted idea (not forgetting that many animal advocates still do not accept it). This means that, at this stage, we need to reach those people most willing to consider new ideas and change. If we can get through to enough of them, the law of supply and demand will ensure that more vegan goods will be made available, making it easier for the next group of people to live vegan.
The trick is knowing - or guessing - what would represent critical mass on this: 30%, 40%, 50%, 60%?
The best way to respond to people who do not "care about animals" is to think that they are not ready yet, that they are thoroughly socialised speciesists, and that they are part of that percentage of the population who will need to be forced to go vegan by the weight of numbers of people already living vegan.