The fast of Ramadaan is obligatory for every adult, sane Muslim who is able to fast. Whoever meets this description, it is haraam for him not to fast without an excuse, because that is clearly going against the command of Allaah and the command of His Messenger (peace and blessings of Allaah be upon him), and it is a transgression against the sanctity of this great month.  

Allaah says (interpretation of the meaning): 

“O you who believe! Observing As-Sawm (the fasting) is prescribed for you as it was prescribed for those before you, that you may become Al-Muttaqoon (the pious)”

[al-Baqarah 2:183] 

“So whoever of you sights (the crescent on the first night of) the month (of Ramadan i.e. is present at his home), he must observe Sawm (fasts) that month, and whoever is ill or on a journey, the same number [of days which one did not observe Sawm (fasts) must be made up] from other days”

[al-Baqarah 2:185]

Fasting becomes obligatory once it is proven that the month of Ramadaan has begun, by the sighting of the new moon or when the month of Sha’baan is completed with thirty days. 

The Prophet (peace and blessings of Allaah be upon him) said: “Islam is built on five (pillars): bearing witness that there is no god but Allaah and that Muhammad is the Messenger of Allaah, establishing regular prayer, paying the zakaah, Hajj, and fasting Ramadaan.” Narrated by al-Bukhaari, 8; Muslim, 16. 

If your question is about delaying the fast until the fourth day with no excuse, then you know that this is haraam and it is not permissible to do this; rather it is a major sin. See question no. 36747. 

If you are delaying the fast for a reason such as sickness or travel, then there is no sin on you for that, but you have to fast as soon as the excuse no longer applies, whether that is on the fourth day or any other, and you also have to make up the days that you did not fast, because of the verse quoted above in which Allaah says “and whoever is ill or on a journey, the same number [of days which one did not observe Sawm (fasts) must be made up] from other days”, i.e., if a sick person or traveller does not fast, then he has to make up the number of days that he missed when Ramadaan is over.