He is most famous as the author of the Shulchan Aruch, the Code of Jewish Law. He was born in Toledo, Spain during the Inquisition years and fled from that country at the age of 4, with his family together with myriads of Jews who were banished from Spain in the year 5252 (1492 CE). His family wandered from city to city, from country to country, not finding a safe haven until they settled in Constantinople (Kushta), Turkey. Yosef was first educated by his father, Rabbi Ephraim, a scholar in his own right, who was later appointed chief rabbi of Nikopol, Bulgaria. Later Rabbi Yosef would quote many teachings he heard from his father. After his father passed away, he grew up in the home of his uncle, Rabbi Yitzchak Caro, who adopted him as his own son. It was soon realized that Yosef was destined for greatness, and even at a young age he was regarded as a great sage and many turned to him for halachic rulings.

He eventually moved from Constantinople to Adrianople, where he married the daughter of a scholar named Rabbi Chaim ibn Albalag. He soon established a Beit Midrash in Adrianople, and at the age of 34 he began to write his monumental commentary Beit Yosef on the entire Arba'ah Turim. Together with his great assiduousness in Torah study, Rabbi Caro lived a somewhat ascetic life of numerous fasts and self-infliction. It was in Adrianople that he met the Kabbalist Rabbi Shlomo Molcho, who was burned at the stake by the church for his "heretical beliefs."

Rabbi Caro was greatly affected by Rabbi Shlomo's charismatic personality and even expressed the wish to die in the same way — al kiddush Hashem (as a holy martyr). It was here, too, that Rabbi Yosef met Rabbi Shlomo Alkabetz, author of the mystical hymn 'Lecha Dodi'. It is possible that one of them introduced Rabbi Yosef to the study of Kabbala. After his first wife died at a young age, he married the daughter of Rabbi Yitzchak Sabba. For a short while he lived in Nikopol, Bulgaria, but decided to make his way to the Holy Land so that he could immerse himself in its sanctity and complete his written works.

Passing through Salonica, he met the great Kabbalist Rabbi Yosef Taitatzak. He continued his journey to the Holy Land via Egypt and eventually settled in Safed. He was soon appointed a member of the rabbinical court of the city in the Beit Din of the famous Rabbi Yaakov Beirav. When the latter re-instituted semicha (official rabbinical ordination), which had been in abeyance for over 11 centuries, Rabbi Yosef was one of the first he ordained. Here, too, Rabbi Caro established a yeshiva and taught Torah to scores of eager students. Among Rabbi Caro's more famous students were the renowned darshan (sermonist) Rabbi Moshe Alshich, the Kabbalist Rabbi Moshe Galanti, and the renowned Kabbalist Rabbi Moshe Cordovero (the Ramak).

When Rabbi Yaakov Beirav, the leading sage of Safed, passed away, Rabbi Yosef Caro was regarded as his successor, and he and Rabbi Moshe of Trani (the Mabit) headed the Rabbinical Court of Safed. In fact, by this time, the Rabbinical Court of Safed had become the central rabbinical court in all of Israel, and indeed of the Diaspora as well. Thus there was not a single matter of national or global importance that did not come to the attention and ruling of the Safed Beit Din. Its rulings were accepted as final and conclusive, and Rabbi Yosef's halachic decisions and clarifications were sought by sages from every corner of the Diaspora. He came to be regarded as the leader of the entire generation.

Although he rarely touched upon Kabbalistic matters and customs in his legal writings, Rabbi Yosef Caro was nevertheless very involved in the study of Kabbala. Together with his close friend Rabbi Shlomo Alkabetz, he set out to explain some of the more difficult passages in the Zohar. In his famous Kabbalistic work Pardes Rimonim, Rabbi Moshe Cordovero cites several innovative Kabbalistic teachings of Rabbi Caro, his master in the revealed teachings of the Torah.

In a dramatic testimonial, Rabbi Shlomo Alkabetz testified that in Salonica, Rabbi Yosef became one of those rare individuals who merited to be instructed by a Magid - a private angelic teacher who revealed to him many Kabbalistic teachings. The Magid exhorted Rabbi Yosef to sanctify and purify himself, and he revealed to him events that would take place in the future. It should be noted that in Shaarei Kedusha, Rabbi Chaim Vital explains that visitation by a Magid is a form of Divine Inspiration (ruach hakodesh).

The teachings of the Magid are recorded in his published work titled Magid Meisharim, although the Chid"a (Rabbi Chaim David Azulai) notes that only about one fiftieth of the manuscript was ever published (see Works). However, in numerous places in magid Meisharim it is stated that, "I am the Mishna that speaks in your mouth," indicating that the Oral Torah itself (of which the Mishna is the fundamental part) spoke within him. (However, these two explanations are not necessarily contradictory — in the merit of the Mishna Rabbi Caro constantly reviewed, he was worthy of an angelic teacher). The Magid promised him that he would have the merit of settling in Israel, and this promise was fulfilled. Another promise, that he would merit to die a martyr's death sanctifying God's Name like Rabbi Shlomo Molcho had merited, did not transpire for an unspecified reason.

The Kabbalistic teachings found in Rabbi Yosef's Magid Meisharim are in the style of Rabbi Moshe Cordovero's Kabbala, rather than the style of Rabbi Yitzchak Luria (the Arizal). Nevertheless, Rabbi Chaim Vital, the chief disciple of the Arizal, extolled the greatness of Rabbi Yosef's soul, saying that it stemmed from the soul of the great Tanna, Rabbi Yehuda bar Ila'i and had an affinity with the souls of Rabbi Shlomo ben Aderet (the Rashb"a), Rabbi Aharon HaLevi (the Ra'ah) and Rabbi Vidal di Tolouse, author of Magid Mishna, an important commentary on Maimonides' Mishneh Torah.

In the year 5324 (1564 CE) Rabbi Yosef's second wife, who had borne him his son Shlomo, died. Following the dictate of the Sages that a man should not live without a wife, he married again, despite his age. His third wife was the daughter of Rabbi Zecharia ben Shlomo Zavasil Ashkenazi, one of the sages of Jerusalem. When he was in his ninth decade, his wife bore him another son, Yehuda. Rabbi Yosef continued to preoccupy himself with Torah study and writing important works and fulfilled his duties as the head of the Rabbinical Court in Safed for the remainder of his extremely productive life.

He passed on to the World of Truth on the 13th day of Nissan, 5335 (1575 CE) at the venerable age of 87. His loss was mourned by the entire Jewish world. The Shelah HaKadosh, writes that one Friday night, Rosh Chodesh Cheshvan in the year 5365 (1605 CE), Rabbi Yosef, thirty years after his passing, appeared in a dream to a certain sage living in Safed. He reported that he saw Rabbi Yosef "sitting on a very majestic throne in the presence of innumerable world-renowned rabbis. His face was shining like the brightness of the sky… and he taught the meditations applicable to holiness."