added login-based workaround to view NSFW content #830

2023-04-08T10:43:28-04:00

PrivacyDevel commented

2023-04-08 10:43:28 -04:00

(Migrated from github.com)

Based on, preferred over and resolves #825
Closes #829 (and closes #302)

Based on, preferred over and resolves #825 Closes #829 (and closes #302)

👍 22 👀 1

roryyamm commented

2023-04-19 12:17:36 -04:00

(Migrated from github.com)

How is this going to work in the end?

PrivacyDevel commented

2023-04-19 14:31:49 -04:00

(Migrated from github.com)

If this pull request got accepted and people updated, they would have the choice to either

leave everything as is and nothing would change
create a Twitter account, login and copy the required information from the sent HTTP headers / cookie list of their browser into the nitter.conf file

If this pull request got accepted and people updated, they would have the choice to either - leave everything as is and nothing would change - create a Twitter account, login and copy the required information from the sent HTTP headers / cookie list of their browser into the nitter.conf file

👍 2 ❤️ 1

OIRNOIR commented

2023-04-27 20:33:46 -04:00

(Migrated from github.com)

I think that you should shield people from viewing NSFW content.

If you feel it is necessary, you may open an issue requesting the user to confirm their age to access NSFW profiles. I don't know if such a thing will be added anyway.
In general, however, Nitter is not a very well-known project among those who should be protected from viewing NSFW content. The vast majority of those people have never even heard of Nitter.
I would urge parents of underage children to parent responsibly instead of urging passionate developers to weigh down their products with chains.

> I think that you should shield people from viewing NSFW content. If you feel it is necessary, you may open an issue requesting the user to confirm their age to access NSFW profiles. I don't know if such a thing will be added anyway. In general, however, Nitter is not a very well-known project among those who should be protected from viewing NSFW content. The vast majority of those people have never even heard of Nitter. I would urge parents of underage children to parent responsibly instead of urging passionate developers to weigh down their products with chains.

bbappserver (Migrated from github.com) reviewed

2023-05-08 00:10:33 -04:00

src/config.nim

					
				@ -47,0 +50,4 @@

				let configPath = getEnv("NITTER_CONF_FILE", "./nitter.conf")

				let (cfg*, fullCfg*) = getConfig(configPath)

@PrivacyDevel This is inconvenient to use when the values can change around if using it in a docker container.
Please change it to

cfg.get("Config", "cookieHeader",  getEnv("cookieHeader", "") )
cfg.get("Config", "xCsrfToken",  getEnv("xCsrfToken", "") )

@PrivacyDevel This is inconvenient to use when the values can change around if using it in a docker container. Please change it to ```nim cfg.get("Config", "cookieHeader", getEnv("cookieHeader", "") ) cfg.get("Config", "xCsrfToken", getEnv("xCsrfToken", "") ) ```

bbappserver (Migrated from github.com) reviewed

2023-05-08 00:20:07 -04:00

src/config.nim

					
				@ -47,0 +50,4 @@

				let configPath = getEnv("NITTER_CONF_FILE", "./nitter.conf")

				let (cfg*, fullCfg*) = getConfig(configPath)

When running docker containers through nice web frontends this makes it easy to change the files without needing to screw around with the twitter conf and getting the cookieHeader all properly escaped for the .conf, because you can just paste it into a text box and it handles the escaping.

When running docker containers through nice web frontends this makes it easy to change the files without needing to screw around with the twitter conf and getting the `cookieHeader` all properly escaped for the `.conf`, because you can just paste it into a text box and it handles the escaping.

somini commented

2023-07-02 20:12:19 -04:00

(Migrated from github.com)

Does this "fix" #919? For public instances this might be hard to implement, but for personal instances, this would be acceptable.

jiatern commented

2023-07-03 10:19:14 -04:00

(Migrated from github.com)

Does this "fix" #919? For public instances this might be hard to implement, but for personal instances, this would be acceptable.

I just set up a private instance, it works well.

> Does this "fix" #919? For public instances this might be hard to implement, but for personal instances, this would be acceptable. I just set up a private instance, it works well.

Mendelrock commented

2023-07-03 13:25:34 -04:00

(Migrated from github.com)

Does this "fix" #919? For public instances this might be hard to implement, but for personal instances, this would be acceptable.

I just set up a private instance, it works well.

Would you mind sharing your ways?
I tried to set it up too, but so far, I still get route errors. Well, to be precise, I can still view individual tweets, but trying to display a user's profile leads to the usual
An error has occured in one of your routes.
Detail: jsony.nim(45) parseHook
asyncfutures.nim(389) read
asyncfutures.nim(389) read
asyncfutures.nim(389) read
asyncfutures.nim(389) read
asyncfutures.nim(389) read
asyncfutures.nim(389) read
Expected { but end reached. At offset: 0
I probably missed a step, but I can't seem to find which :'(

> > Does this "fix" #919? For public instances this might be hard to implement, but for personal instances, this would be acceptable. > > I just set up a private instance, it works well. Would you mind sharing your ways? I tried to set it up too, but so far, I still get route errors. Well, to be precise, I can still view individual tweets, but trying to display a user's profile leads to the usual `An error has occured in one of your routes.` `Detail: jsony.nim(45) parseHook` `asyncfutures.nim(389) read` `asyncfutures.nim(389) read` `asyncfutures.nim(389) read` `asyncfutures.nim(389) read` `asyncfutures.nim(389) read` `asyncfutures.nim(389) read` `Expected { but end reached. At offset: 0` I probably missed a step, but I can't seem to find which :'(

jiatern commented

2023-07-03 21:46:22 -04:00

(Migrated from github.com)

Would you mind sharing your ways?

I am not sure what you missed, but these are my steps:

Clone PrivacyDevel's fork https://github.com/PrivacyDevel/nitter, I use the master branch
Install all dependencies as instructed (libpcre, libsass, redis, nimble)
Update configurations in nitter.conf, ensure ct0, auth_token and xCsrfToken are also updated
Build nitter by following the guidelines
Run redis, and then run nitter

I am able to access individual user profiles as well.

> Would you mind sharing your ways? I am not sure what you missed, but these are my steps: 1. Clone PrivacyDevel's fork https://github.com/PrivacyDevel/nitter, I use the master branch 2. Install all dependencies as instructed (libpcre, libsass, redis, nimble) 3. Update configurations in nitter.conf, ensure `ct0`, `auth_token` and `xCsrfToken` are also updated 4. Build nitter by following the guidelines 5. Run redis, and then run nitter I am able to access individual user profiles as well.

fnxweb commented

2023-07-06 08:14:49 -04:00

(Migrated from github.com)

It's not working for me, but I think it may be more fundamental, as I've not built nitter before. For a start, the Nitter "front page" is stuffed, with over-sized images, suspect some CSS is AWOL.
Then if I try to view a user, I actually see 404s coming back from Twitter – is that a symptom of not having working ct0/auth_token/XCsrfToken values? I have double checked that what is in the Chrome dev. network data is what I have in the config file.

404 Not Found: --- url: https://api.twitter.com/graphql/somecodehere/UserByScreenName?variables=...etc

It's not working for me, but I think it may be more fundamental, as I've not built nitter before. For a start, the Nitter "front page" is stuffed, with over-sized images, suspect some CSS is AWOL. Then if I try to view a user, I actually see 404s coming back from Twitter – is that a symptom of not having working ct0/auth_token/XCsrfToken values? I have double checked that what is in the Chrome dev. network data is what I have in the config file. 404 Not Found: --- url: https://api.twitter.com/graphql/somecodehere/UserByScreenName?variables=...etc

fnxweb commented

2023-07-06 08:21:19 -04:00

(Migrated from github.com)

The CSS is probably unrelated to the main functionality not working, but if I fetch http://localhost:8080/css/style.css I actually get an HTML Nitter error not the CSS file.

The CSS is _probably_ unrelated to the main functionality not working, but if I fetch http://localhost:8080/css/style.css I actually get an HTML Nitter error not the CSS file.

fnxweb commented

2023-07-06 08:31:55 -04:00

(Migrated from github.com)

CSS fixed, I had neglected the nimble scss (which by its name is plainly related!)

Still get 404 out of Twitter though :(

CSS fixed, I had neglected the **nimble scss** (which by its name is plainly related!) Still get 404 out of Twitter though :(

PrivacyDevel commented

2023-07-06 10:03:46 -04:00

(Migrated from github.com)

CSS fixed, I had neglected the nimble scss (which by its name is plainly related!)

Still get 404 out of Twitter though :(

Does it work when you are using my fork instead?
Did you accidentally add spaces to those lines where there shouldn't be any?
Did you make any other big changes to the conf file?
What's the version string on the /about page?

> CSS fixed, I had neglected the **nimble scss** (which by its name is plainly related!) > > Still get 404 out of Twitter though :( Does it work when you are using my fork instead? Did you accidentally add spaces to those lines where there shouldn't be any? Did you make any other big changes to the conf file? What's the version string on the /about page?

fnxweb commented

2023-07-06 10:48:35 -04:00

(Migrated from github.com)

I started from a git clone of nitter, created a new branch and pulled in your nsfw pull req. I've not yet tried a direct clone of your branch, but I will.

I copied my working nitter.conf and added the two new auth fields which I'll partially paste below. I'll do a more detailed diff of that and your updated example in case I missed something else. I do have your changes locally, and have rebuilt. Oh. But I added some extra debug to print the new fields and that's not coming out, either meaning my config is not being picked up, or it's not somehow rebuilt properly?

.. in [Config] section after the comment describing them
cookieHeader = "ct0=e5...ff; auth_token=bb...42"
xCsrfToken = "e5...ff"

That's interesting, the ct0 and Csrf are the same, I'd not noticed that. Expected? I'm sure I double checked them.

Version 2023.07.06-bb7e239

I started from a git clone of nitter, created a new branch and pulled in your nsfw pull req. I've not yet tried a direct clone of your branch, but I will. I copied my working nitter.conf and added the two new auth fields which I'll partially paste below. I'll do a more detailed diff of that and your updated example in case I missed something else. I do have your changes locally, and have rebuilt. Oh. But I added some extra debug to print the new fields and that's not coming out, either meaning my config is not being picked up, or it's not somehow rebuilt properly? .. in [Config] section after the comment describing them cookieHeader = "ct0=e5...ff; auth_token=bb...42" xCsrfToken = "e5...ff" That's interesting, the ct0 and Csrf are the same, I'd not noticed that. Expected? I'm sure I double checked them. Version [2023.07.06-bb7e239](https://github.com/zedeus/nitter/commit/bb7e239)

fnxweb commented

2023-07-06 11:00:59 -04:00

(Migrated from github.com)

It is rebuilding OK, but my debug is bogus, possibly due to not understanding nim, so I can't verify what it's using, if anything, for the auth headers. Tried to add echos to proc fetch. Builds but nothing printed, although I can adjust the error output that I get.

fnxweb commented

2023-07-06 11:16:07 -04:00

(Migrated from github.com)

OK, a raw build of your branch behaves differently, I now get the error occured in one of your routes that @Mendelrock reported above. And the debug still reports a 404 coming back from the API URL. I think that error may be a symptom of the 404, as Z's master now has a JSON parser/exception fix that you haven't yet got, I think (but I have in my other build as I merged nsfw onto his latest).

So I still have some issue somewhere.

OK, a raw build of your branch behaves differently, I now get the _error occured in one of your routes_ that @Mendelrock reported above. And the debug still reports a 404 coming back from the API URL. I think that error may be a symptom of the 404, as Z's master now has a JSON parser/exception fix that you haven't yet got, I think (but I have in my other build as I merged nsfw onto his latest). So I still have some issue somewhere.

👍 1

fnxweb commented

2023-07-06 11:40:46 -04:00

(Migrated from github.com)

Right, not sure what I was doing wrong, but I do get debug now, and upon the 404 I print my auth tokens and they are as I expect. But I can see zero evidence of them being used. The only time they are picked up is in the proc fetch (as above) and debug does not show this being called. I get errors from proc fetchRaw.

It is a bit more complex as well. If I search for the user (BungieHelp FWIW) I do get results. But if then click on a user name (that or any others), it comes back not found (well, a 404, which Z's master has a fix-ish for, so "not found" instead of the above routes error message).

I'm wondering: does your patch only add auth to the points where you found it was previously required (like search) and it is not present in the path that is used for a user view / feed (and so that now fails)?

Right, not sure what I was doing wrong, but I do get debug now, and upon the 404 I print my auth tokens and they are as I expect. But I can see zero evidence of them being used. The only time they are picked up is in the proc fetch (as above) and debug does not show this being called. I get errors from proc fetchRaw. It is a bit more complex as well. If I *search* for the user (_BungieHelp_ FWIW) I do get results. But if then click on a user name (that or any others), it comes back _not found_ (well, a 404, which Z's master has a fix-ish for, so "not found" instead of the above routes error message). I'm wondering: does your patch only add auth to the points where you found it was previously required (like search) and it is not present in the path that is used for a user view / feed (and so that now fails)?

fnxweb commented

2023-07-06 11:48:23 -04:00

(Migrated from github.com)

Confirmed! This fixes user feeds:

diff --git a/src/apiutils.nim b/src/apiutils.nim
index bc4b498..35de0b6 100644
--- a/src/apiutils.nim
+++ b/src/apiutils.nim
@@ -122,6 +122,12 @@ proc fetch*(url: Uri; api: Api; additional_headers: HttpHeaders = newHttpHeaders
       raise rateLimitError()
 
 proc fetchRaw*(url: Uri; api: Api; additional_headers: HttpHeaders = newHttpHeaders()): Future[string] {.async.} =
+
+  if len(cfg.cookieHeader) != 0:
+      additional_headers.add("Cookie", cfg.cookieHeader)
+  if len(cfg.xCsrfToken) != 0:
+      additional_headers.add("x-csrf-token", cfg.xCsrfToken)
+
   fetchImpl(result, additional_headers):
     if not (result.startsWith('{') or result.startsWith('[')):
       echo resp.status, ": ", result, " --- url: ", url

Confirmed! This fixes user feeds: ``` diff --git a/src/apiutils.nim b/src/apiutils.nim index bc4b498..35de0b6 100644 --- a/src/apiutils.nim +++ b/src/apiutils.nim @@ -122,6 +122,12 @@ proc fetch*(url: Uri; api: Api; additional_headers: HttpHeaders = newHttpHeaders raise rateLimitError() proc fetchRaw*(url: Uri; api: Api; additional_headers: HttpHeaders = newHttpHeaders()): Future[string] {.async.} = + + if len(cfg.cookieHeader) != 0: + additional_headers.add("Cookie", cfg.cookieHeader) + if len(cfg.xCsrfToken) != 0: + additional_headers.add("x-csrf-token", cfg.xCsrfToken) + fetchImpl(result, additional_headers): if not (result.startsWith('{') or result.startsWith('[')): echo resp.status, ": ", result, " --- url: ", url ```

👍 4

y-bonfire commented

2023-08-31 01:23:30 -04:00

(Migrated from github.com)

I tried this pr today. While RSS retrieval is functioning, it seems that the search functionality is not working properly.
1234b9ba94
The same situation?
https://github.com/zedeus/nitter/issues/983#issuecomment-1694706348

I tried this pr today. While RSS retrieval is functioning, it seems that the search functionality is not working properly. 1234b9ba940d689789a02876219a408de400a44f The same situation? https://github.com/zedeus/nitter/issues/983#issuecomment-1694706348 ![image](https://github.com/zedeus/nitter/assets/80713642/4807ff1f-fe6b-48ac-8944-e35efd18d7be)

zedeus commented

2025-01-25 19:59:17 -05:00

(Migrated from github.com)

This won't be needed anymore