Yeah...apparently that setting can only be turned on when you have an internet connection ie, scenario's:
1.) I am connected and then select "go offline", then after that I can disconnect my internet and carry on playing offline.
2.) I had internet and then moved and now have no internet, now when I turn on my Xbox and put a game in, it says "This xbox needs to be connected to the internet to play games, please connect to the internet and sign in". Now, when I go into settings and select "go offline" it doesn't take affect, it's almost like the X1 has to tell a server to set it to offline mode or something. Still not a 100% sure, but this is the answer I seem to get 90% of the time