I have a 1997 Mazda 626 that doesn't have heat changed radiator and the water pump already.

Unless the core is dumping fluid all over the floor boards, you don't need a new core. It is most likely due to the switch when you turn the car for A/C to Heat. The switch is probably bad. I assume your blower motor is working. You can go on Ebay and search under 97 Mazda climate control unit.

Look under hood and locate the two heater hoses going thru the firewall below the front windshield. With vehicle warmed up, heater on high and vehicle idling feel these two hoses. Are both hot? If so you probably have a blend door issue under the dash. Is one hot and the other one cold or luke warm? If so you probably have a stopped up heater core. Are both cold or luke warm? If so you probably have a stuck open thermostat. Hope this helps.

One heater hose is like not really hot and the other one is stone cold. Sounds like the heater core is plugged.Have you ever had any luck flushing this out?

I have at times. I removed the heater hoses and used a water hose to blow thru the heater core. Flush thru one hose and then thru the other. Did several times till I had good flow going both ways. Use caution not to get water all over the engine. I placed longer pieces of hose onto the core so that the water would not splash my engine as I did this.