Genome-wide association studies (GWAS) have revolutionized human genetics, allowing researchers to identify thousands of disease-related genes and possible drug targets. However, case-control status does not account for the fact that not all controls may have lived through their period of risk for the disorder of interest. This can be quantified by examining the age-of-onset distribution and the age of the controls or the age-of-onset for cases. The age-of-onset distribution may also depend on information such as sex and birth year. In addition, family history is not routinely included in the assessment of control status. Here we present LT-FH++, an extension of the liability threshold model conditioned on family history (LT-FH), that jointly accounts for age-of-onset and sex, as well as family history. Using simulations, we show that, when family history and the age-of-onset distribution are available, the proposed approach yields large power gains over both LT-FH and genome-wide association study by proxy (GWAX). We applied our method to four psychiatric disorders available in the iPSYCH data, and to mortality in the UK Biobank, finding 20 genome-wide significant associations with LT-FH++, compared to 10 for LT-FH and 8 for a standard case-control GWAS. As more genetic data with linked electronic health records become available to researchers, we expect methods that account for additional health information, such as LT-FH++, to become even more beneficial.