{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import praw\n", "import pandas as pd\n", "from Keys import reddit_keys\n", "\n", "reddit= praw.Reddit(client_id=reddit_keys['client_id'],\t\t # your client id\n", "\t\t\t\t\tclient_secret=reddit_keys['client_secret'],\t # your client secret\n", " usernme = reddit_keys['usernme'], #profile username\n", " password = reddit_keys['password'], #profile password\n", "\t\t\t\t\tuser_agent=reddit_keys['user_agent'])\t # your user agent\n", "\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "from praw.models import MoreComments\n", "from transformers import pipeline\n", "from transformers import DistilBertTokenizerFast\n", "classifier = pipeline(\"sentiment-analysis\", model=\"michellejieli/NSFW_text_classifier\")\n", "\n", "input_url = \"https://www.reddit.com/r/europe/comments/r0hthg/sweden_is_taking_the_lead_to_persuade_the_rest_of/\"\n", "\n", "def extract_comments(input_url):\n", " submission = reddit.submission(url=input_url)\n", " #posts_dict = {\"Post text\":[],}\n", " posts_dict = {\"Post text\":[], \"class\": []}\n", " for top_level_comment in submission.comments:\n", " if isinstance(top_level_comment, MoreComments):\n", " continue\n", " \n", " posts_dict[\"Post text\"].append(top_level_comment.body)\n", " posts_dict[\"class\"].append(classifier(top_level_comment.body)[0]['label'])\n", " df = pd.DataFrame(posts_dict)\n", " return df " ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Post textclass
0Does… does that mean GPU’s might become afford...SFW
1But I use my mining rig for interior heating. ...SFW
2I approve.\\n\\nAlthough have they suggested how...SFW
3You can't ban cryptocurrencies. Just like you ...SFW
4But still no nuclear plantsSFW
5https://twitter.com/ercwl/status/1457114531314...NSFW
6Yes, \"we\" are missing the goal. Maybe stop sub...NSFW
7here is something complete insane to think abo...NSFW
8Put a tax carbon on it, it will elimanate the ...SFW
9Banning crypto mining will only cause it to mo...SFW
10Can someone explain how this is possible?SFW
11The average knowledge on crypto in here is ver...NSFW
12Howbout u ban Facebook instead?NSFW
13Good. Blockchain is useful but so many despera...NSFW
14I don't particularly like crypto or it's fans,...SFW
15I was thinking of getting rid of my RTX 2070s,...SFW
16Total waste of time. As seen following the Chi...SFW
17It's actually shocking how uneducated people a...NSFW
18Totally doesn’t have anything to do with money...NSFW
19Sweden's biggest electric company Vattenfall s...NSFW
20BASEDSFW
21Excellent news.\\n\\nIt consumes vast quantities...NSFW
22sweden want to fuck with danemark and his cryp...NSFW
23They should ban gold too, mining gold uses 10x...NSFW
24Great news. Cryptocurrencies are an embarrassi...NSFW
25This is BS, it doesn't make sense.NSFW
26I see no problem with this, the amount of ener...SFW
27[deleted]SFW
28\"I'm running my graphics card at full load so ...SFW
29The market is moving from POW to POS which is ...SFW
30And exactly how are they gonna do that?\\n\\nPir...SFW
31Yeah because that's what's the main cause of o...SFW
32Banning mining without banning transactions an...NSFW
33Crypto seems to have some side effects, who wo...SFW
34Crypto currency is the new tulips. So hot righ...NSFW
35Fuck yeah, ban them all! Easiest climate check...NSFW
36Ban fossil fuels instead. What a useless measu...NSFW
37Please no. I earn 200 euros a month from it.SFW
38Swedens state owned power company rejects this...NSFW
39Great not only we had Chinese crypto miners se...SFW
40Let's ban the currency that is somewhat indepe...NSFW
41Is that really that much of a problem with ene...NSFW
42How about instead of banning it, we make it so...SFW
43The electric footprint of crypto won't go up. ...SFW
44Lets make playing a demanding game more than t...NSFW
45How does cryptocurrency change the climate? I ...SFW
46Load ze Sweden FUDSFW
47[removed]NSFW
48Such pathetic virtue-signaling.\\n\\nIts no diff...NSFW
49shame on swedenNSFW
50-And now, NEWS FOR DUMBASSES, Bob?\\n\\n-Yes Bri...NSFW
51dumbNSFW
52Wow, Sweden is such a dictatorial shithole cou...NSFW
53talking about that, a full ban might be unlike...NSFW
\n", "
" ], "text/plain": [ " Post text class\n", "0 Does… does that mean GPU’s might become afford... SFW\n", "1 But I use my mining rig for interior heating. ... SFW\n", "2 I approve.\\n\\nAlthough have they suggested how... SFW\n", "3 You can't ban cryptocurrencies. Just like you ... SFW\n", "4 But still no nuclear plants SFW\n", "5 https://twitter.com/ercwl/status/1457114531314... NSFW\n", "6 Yes, \"we\" are missing the goal. Maybe stop sub... NSFW\n", "7 here is something complete insane to think abo... NSFW\n", "8 Put a tax carbon on it, it will elimanate the ... SFW\n", "9 Banning crypto mining will only cause it to mo... SFW\n", "10 Can someone explain how this is possible? SFW\n", "11 The average knowledge on crypto in here is ver... NSFW\n", "12 Howbout u ban Facebook instead? NSFW\n", "13 Good. Blockchain is useful but so many despera... NSFW\n", "14 I don't particularly like crypto or it's fans,... SFW\n", "15 I was thinking of getting rid of my RTX 2070s,... SFW\n", "16 Total waste of time. As seen following the Chi... SFW\n", "17 It's actually shocking how uneducated people a... NSFW\n", "18 Totally doesn’t have anything to do with money... NSFW\n", "19 Sweden's biggest electric company Vattenfall s... NSFW\n", "20 BASED SFW\n", "21 Excellent news.\\n\\nIt consumes vast quantities... NSFW\n", "22 sweden want to fuck with danemark and his cryp... NSFW\n", "23 They should ban gold too, mining gold uses 10x... NSFW\n", "24 Great news. Cryptocurrencies are an embarrassi... NSFW\n", "25 This is BS, it doesn't make sense. NSFW\n", "26 I see no problem with this, the amount of ener... SFW\n", "27 [deleted] SFW\n", "28 \"I'm running my graphics card at full load so ... SFW\n", "29 The market is moving from POW to POS which is ... SFW\n", "30 And exactly how are they gonna do that?\\n\\nPir... SFW\n", "31 Yeah because that's what's the main cause of o... SFW\n", "32 Banning mining without banning transactions an... NSFW\n", "33 Crypto seems to have some side effects, who wo... SFW\n", "34 Crypto currency is the new tulips. So hot righ... NSFW\n", "35 Fuck yeah, ban them all! Easiest climate check... NSFW\n", "36 Ban fossil fuels instead. What a useless measu... NSFW\n", "37 Please no. I earn 200 euros a month from it. SFW\n", "38 Swedens state owned power company rejects this... NSFW\n", "39 Great not only we had Chinese crypto miners se... SFW\n", "40 Let's ban the currency that is somewhat indepe... NSFW\n", "41 Is that really that much of a problem with ene... NSFW\n", "42 How about instead of banning it, we make it so... SFW\n", "43 The electric footprint of crypto won't go up. ... SFW\n", "44 Lets make playing a demanding game more than t... NSFW\n", "45 How does cryptocurrency change the climate? I ... SFW\n", "46 Load ze Sweden FUD SFW\n", "47 [removed] NSFW\n", "48 Such pathetic virtue-signaling.\\n\\nIts no diff... NSFW\n", "49 shame on sweden NSFW\n", "50 -And now, NEWS FOR DUMBASSES, Bob?\\n\\n-Yes Bri... NSFW\n", "51 dumb NSFW\n", "52 Wow, Sweden is such a dictatorial shithole cou... NSFW\n", "53 talking about that, a full ban might be unlike... NSFW" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mThe Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click here for more info. View Jupyter log for further details." ] } ], "source": [ "extract_comments(input_url)\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.4 (main, Nov 29 2022, 20:00:25) [GCC 9.4.0]" }, "orig_nbformat": 4, "vscode": { "interpreter": { "hash": "3ad933181bd8a04b432d3370b9dc3b0662ad032c4dfaa4e4f1596c548f763858" } } }, "nbformat": 4, "nbformat_minor": 2 }