Sparta Prague B
Born in Rychnov nad Kněžnou, Czechoslovakia, Holý signed for Czech side Sparta Prague in 2010, at the age of eighteen, having previously been with Hradec Králové since the age of fourteen. He made his career league debut for AC Sparta Prague B on 11 June 2011 in a Czech 2. Liga 1–2 home loss against FK Varnsdorf. He made 11 appearances during the 2011–12 season. During his time with the club he was loaned to Czech sides Vlašim, Viktoria Žižkov and Fastav Zlín. At Zlín he made 20 appearances in the Czech First League, the top level of the Czech football league system.
Holý signed a two and a half year contract with EFL League One side Gillingham on 20 January 2017. Commenting on the signing, Gillingham head coach Adrian Pennock said "He is a good size, a good presence, he kicks it well, handles it well and it is good competition for Stuart Nelson, that’s what we need". He made his debut for the side on 25 March 2017 in the home 0–1 loss to Peterborough United. He went on to make 6 appearances during the second half of the 2016–17 season.
He kept his place for the 2018–19 season, making 52 appearances over the course of the season, keeping 15 clean sheets. He was offered a new contract by Gillingham at the end of the 2018–19 season but he ultimately declined the contract.
On 24 May 2019, Holý agreed to join Ipswich Town on a two-year deal upon the expiry of his Gills contract. He was given the number 1 shirt upon joining the club. He made his debut for the club on 3 August 2019, keeping a clean sheet in a 1–0 away win over Burton Albion at the Pirelli Stadium, on the opening day of the 2019–20 season. He kept 6 clean sheets in the first 10 league games of the season, including a run of 5 consecutive clean sheets in the league, equaling a club record. On 26 November, Holý saved a penalty in a 0–0 home draw with Wycombe Wanderers at Portman Road.