In 1896, gun founding is a bloody complicated business. To be able to sustain an arms industry even making relatively simple pieces (field guns, light naval armament), you will require (either as import suppliers or on hand):
steelworks with modern blast furnaces and the ability to refine and precisely alloy steel
the metallurgical industry to supply the additives required
machine shops that can turn out rifled barrels, breech blocks, shells and mechanical parts
the infrastructure to produce and maintain those machine tools and power the shops, either electrically or by mechanical transmission
cokeries to feed your furnaces
a fairly basic chemical industry to produce explosives and various other substances required for fuses and lubricants
coal and iron mines
And not to forget: rubber works, a (vegetable) oil refining industry and wood workshops.
This is not beyond the ability of even a smallish country - the Netherlands and Belgium had arms industries. But it is hardly trivial. THe big problem to overcome will not be the top tier of expertise. Nineteenth-century Europeans tended to travel widely, and largely unopposed and unhassled. If an engineer or gunner wanted to go to China, Siam or Hawaii, he could. The main problem will be importing or creating the mid-level skill. Japan in 1896 still could not produce a lot of the more demanding things (like large-calibre guns and capital ships) at home, and they had been modernising for five decades by that point. If you are not born into an industrial infrastructure, it is very hard to acquire the necessary skills and mental habits. You'll need veterans of heavy industry with a Kiplingesque "charm for making riflemen from mud" to pull it off.
Exactly. People tend to forget that something like an arms industry does not exist in a vacuum; it is part of an industrial ecosystem, without which it could not function. For your intended country to support an arms industry it needs to have the rest of the ecosystem as well; you would have to do a Meiji there as well.
The best candidate for this is China, of course. They have the population base, the resources, and some industry already; if you get rid of the reactionaries in charge (i.e. the Dowager Empress and her supporters) and put someone more progressive in charge to push things along you could have the Chinese be a much more serious opponent than IOTL.
Next possibility would be Korea. Unfortunately for them the Japanese already have designs on them, and being right next door, so to speak, makes them an easy target. If the Koreans can overcome their dislike of the Japanese (and the Japanese can be persuaded to treat them as something approaching equals) then perhaps they can come to a mutually beneficial arrangement, wherein Korea supplies raw materials in exchange for Japanese manufactures, for instance. Once they acquire industry of their own they can arrange to be treated as a junior partner in a Japanese-Korean alliance, which will benefit both parties greatly. The butterflies from this will be very large and numerous ...
Next possibility is the Philippines. They have the advantages of a large population and a relatively isolated position, which means they won't need a large military and can devote proportionately more to development. If they can throw off the Spanish yoke and receive US and/or European aid they can become a much more serious obstacle to Japanese agression.
Southeast Asia is at this time firmly under the thumbs of the British and French, who are not about to let the natives have arms, let alone an arms industry. The same is even truer of Indonesia and the Dutch.
Australia is a slightly different case. As a Dominion it has the freedom to develope arms industries if it so desires. The difficulty here is the small population base, which limits the size of its indutries. They will end up producing small quantities of high quality arms, as IOTL. IOTL they made as much of a contribution toward fighting the Japanese as could be expected given their resources; much more than that is probably beyond their capability.
