He hailed from Yunzhong 雲中 (modern Hohhot, Inner Mongolia). Because of his brave conduct he was made regional inspector (cishi 刺史) of the province of Bingzhou 幷州. In the late years of Emperor Ling's 漢靈帝 (r. 167-189) reign he was made acting commander (jia sima 假司馬) of the Guard of the Western Garden 西園軍. When this unit was dissolved the regent He Jin 何進 sent him back to his home province where he was ordered to recruit troops.
When Dong Zhuo 董卓 usurped the power in the central government Zhang Yang declared his base, the commandery of Shangdang 上黨, independent. Dong Zhuo made him General establishing propriety (jianyi jiangjun 建義將軍) and governor (taishou 太守) of the commandery of Henei 河內. After the death of the tyrant, Zhang Yang supported Yang Feng 楊奉 and Dong Cheng 董承 to bring Emperor Xian 漢獻帝 (r. 189-220) back to the capital Luoyang 洛陽. For these services Zhang Yang was appointed Minister of War (dasima 大司馬).
In 195 the warlord Cao Cao 曹操 attacked Lü Bu 呂布, who was befriended to Zhang Yang. Because he was unable to support Lü Bu, Zhang Yang deployed his troops to distract Cao Cao's troops from Lü Bu. During this campaign Zhang Yang was killed by his own general Yang Chou 楊醜.