It looks like you have a custom master page that is missing much of the core structure of a master page. In particular, an out of the box master page responds to a Query String value of "?IsDlg=1" to control if master page content is displayed in SharePoint dialog boxes along with primary page content.
To see this, go to a site without this custom master page and visit a list or library page. Edit the URL and, if there's no Query String (i.e. no "?" in the URL, add "?IsDlg=1" (no quotes), or if there is a query string add "&IsDlg=1" (no quotes). Notice the difference in the displayed content.
Something is missing from your custom master page. Either the lack of standard CSS classes or missing JavaScript references.
Can you reset the master page back to one of the out of the box master pages and check to see if everything is now working.
As the the Site Settings, someone must of added an IFRAME or a Page Viewer web part to the home page and set the URL to the Site Settings page.