The attack injures 21 and damages infrastructure facilities, local officials report
An "active discussion" in government is ongoing, with Foreign Minister Liz Truss said to be in favour of the boycott, the newspaper said.
Under one option, British ministers would not attend the Games, although the ambassador to China would, Times added.
Five Conservative politicians wrote to Johnson urging him to ban any official UK diplomatic representation at the Winter Olympics in February, according to the report.
President Joe Biden confirmed on Thursday that the United States is considering a diplomatic boycott of the Beijing Olympics.