Tuesday, November 18, 2014

TBitArray

   I looked around on the internet to see if there was any way for me to store individual bits in an array. The BitArray class exists but it states that it uses booleans to represent bits. I then went to look at how large booleans are. MSDN says it varies and on several threads the size ranges from 2-4 bytes. I haven't done any testing to see how much the actual size of booleans when used in the BitArray class but I went a head and threw together this TBitArray class anyways. Bits are stored in a byte array but each bit in the array is given an index instead of each byte given an index. 8bits = 1byte array, 9bits = 2byte array, 16bits = 2byte array, 17bits = 3byte array etc...


As always this might not be the best approach or implementation, if you have any issues or suggestions please leave a comment.

Sunday, October 12, 2014

XwitchBot2 [BETA] Release

This is the BETA release for XwitchBot2. It implements most if not all intended functionality (at least ones I remembered). Plugins will come slow and the Raffle plugin included is a dummy plugin to test plugin system. If any issues are found please comment.

Download: https://www.mediafire.com/folder/jze9zh71g0du5//XwitchBot2
Only the ZIP file is needed.


UPDATE - Oct 14, 2014:
Fixed a bug in Chat menu option not reverting to dark gray.
Fixed a bug in the plugins selection menu that causes crash.
Fixed a bug in several basic twitch IRC commands that causes crash.
Fixed a bug in plugins command processing that causes crash.
Fixed a bug preventing application from closing properly when expected.
Corrected message username to display "XwitchBot" instead of the current Nick when expect.
Added Restore option to system tray icon.
Added Command Line Argument to set bot's OAuth.
Added Command Line Argument to set owner's OAuth.
Added Command Line Argument to set server.
Added Command Line Argument to set server port.
Added Command Line Argument to auto connect.
Added Functional "Retarded" plugin, with dummy UI.
Changed Title name to state "Nick" and "Channel".


To make plugins use the Core.dll although this is only BETA the Core.dll might change before final version (though unlikely).
Here is a screenshot:


Here is a basic starting point for making a plugin:
Start a new project.
Select Class Library.
Create a new User Control in the project, call it UI.
Set the Size property to 842, 488.
Lock the control.
Add a Panel to the form.
Change the Panel's Dock property to Fill.
In the <YourPluginName>.vb file follow the below example

Sunday, August 10, 2014

McCollough Effect

Made a little program because I got bored. It induces the McCollough Effect.
While the wiki says you should look at each image several seconds at a time it didn't specify so I just used 5 seconds intervals.
You will first be shown a black and white image with horizontal and vertical grating for 10 seconds. You will then be shown a red with horizontal grating, then a green one with vertical grating each being shown for 5 seconds for a total of 3 minutes. The same black and white image from the beginning will be shown again at the end. If the Effect was successful the image should look slightly different now.
To exit the application at any time press escape.
PS: The application runs in full screen.

Download: http://www.mediafire.com/download/niurrwq3e92s3ac/McCollough_Effect.exe

Thursday, August 7, 2014

Image2GVD

Here is an application requested by Ada and maybe a few other on IRC. At first I didn't want to do it since I am lazy and didn't know anything about it. After they linked me to the page I thought I can do this though I cant test etc. I eventually started it and here is the completed app (though I personally never tested but Ada has tested and says its working). It generates a GVD file used for PlayView on the PlayStation3 and possibly other devices.

PS: Thanks Ada for explaining the GVD structure etc.

Download: http://www.mediafire.com/download/k1n5kfmc2wkm7kx/Image2GVD.exe
Download for those on XP : http://www.mediafire.com/download/rg7cdyvz0w6yviv/Image2GVD-XP.exe

Usage: Normally run the executable and follow onscreen
                     or
             Drag image into CMD when asked.

             Running the application and the image as argument also works

             Generated GVD will be created in the same directory as the source image.

             Images must be 3840x2160 for now, might make custom later.

As always if you have problems or suggesting please leave a comment.

Tuesday, June 3, 2014

XwitchBot2 Started

Update: Starting over. I goofed and messed up the bot lol but not all is lost I can bring over many of the things from this version. I will be working slowly to ensure no memory leaks and no UI hangups occur. I will also improve on the plugin system giving it more control etc.

For those still using XwitchBot, I am working on XwitchBot2 at the moment. I will post updates on its progress here. The idea is that XwitchBot while pretty functional in itself lacks some capabilities and making plugins is easy but not extremely easy. The UI could also use some work.
Progress:
 Core = DONE
 Plugin interface = DONE
 Chat UI = DONE
 UserList = DONE
 Options UI = DONE
 Plugins UI = DONE
 Donation UI = DONE
 About UI = DONE
 TwitchAPI = DONE
 IRC component = DONE
 Plugin Manager = DONE
 MessageProcessor = DONE
 Greetings = Redone ~95%
 ProgrammableKeySupport = DONE
 InChat Basic Twitch Commands = DONE
 Timestamps = DONE
 OAuthGeneration Helper = DONE
 ....if more components are added I will update this list including their progress.
The percentages here indicate how close a component is to intended functionality and not an indication of completeness.

UPDATE 8/11/2014
Thought I would upload a video showing what the bot is like at the moment. Its not complete but its very close to it. So here is a video of it doing its thing:

Welp thats that, should be done soon, if me getting kicked out doesn't get in the way.


UPDATE 8/10/2014
Separated the bot and core for use with plugins and implemented the plugin manager.

UPDATE 8/9/2014
Removed the Staff and Admin lists in chat (find them unnecessary).
Fixed random disconnect (had the close events in the paint event handler :P, put it in the mouseclick event now)
Added a reconnect feature just in case of the connection is cut (will not work if internet connection itself is lost or unavailable)
Made the Setup page only show upon start up (no more disconnect feature)
Forgot to mention I implemented all basic IRC commands available to twitch (/timeout /ban etc)
Cleaned up some things.

UPDATE 7/23/2014 (much done today)
I got to DONE status for many of the components on the list and progressed further in some others as well.
Currently working on improving the IRC component though. It's still 100% functional so its DONE status will remain as it. I am making it more robust and processing received messages. I will also be removing the staff and admin areas in the chat as I find them unnecessary.

UPDATE 6/26/2014 (IRC component in place and tested):

UPDATED UI, It is far from complete but here is a sample of the working UI thus far:

[OLD]Well the bot is nearing completion and here is what it looks like for those curious:

Friday, May 30, 2014

Custom OAuth




UPDATE July 26, 2017: With additions and updates to twitch there are now more scopes possible when generating an OAuth, these new ones have been added per v5 of the TwitchAPI. Scopes are now sorted Alphabetically. chat_login scope is now an option instead of being included always.

If you are logged into Twitch/Justin an OAuth will be generated for that account. To ensure nothing goes wrong check
Twitch/Justin or simply go and logoff, you will be asked to login to the account you want to generate an OAuth for.

channel_check_subscription: Read access to check if a user is subscribed to your channel.
channel_commercial: Access to trigger commercials on channel.
channel_editor: Write access to channel metadata (game, status, etc).
channel_feed_edit: Add posts and reactions to a channel feed.
channel_feed_read: View a channel feed.
channel_read: Read access to non-public channel information, including email address and stream key.
channel_stream: Ability to reset a channel's stream key.
channel_subscriptions: Read access to all subscribers to your channel.
chat_login: Log into chat and send messages.
collections_edit: Manage a user’s collections (of videos).
communities_edit: Manage a user’s communities.
communities_moderate: Manage community moderators.
user_read: Read access to non-public user information, such as email address.
user_blocks_edit: Ability to ignore or unignore on behalf of a user.
user_blocks_read: Read access to a user's list of ignored users.
user_follows_edit: Access to manage a user's followed channels.
user_subscriptions: Read access to subscriptions of a user.
viewing_activity_read: Turn on Viewer Heartbeat Service ability to record user data.
openid: Use OpenID Connect authentication.



While working with the Twitch API I noticed that scopes are like the creditials an access card has. At first I figured any generated OAuth key should have complete access to my own room (channel) but I was sadly mistaken. An OAuth key must be generated with the specified creditials and this is absolute. So comes the next task, how do I go about generating OAuths with the proper/needed scope? The most common and known OAuth generator only generates OAuths for IRC login for chat with only the scope "chat_login". I thought is there a way to modify this generator so that it will request a new key with the added scopes. I looked on the documented page and it states that you can specify the scope in the URL. I went back to the generator and load and behold at the end of that URL was "&scope=chat_login". The TwitchAPI documentation says I can add scopes seperating them with a plus sign "+" and so I tested it. I changed the end of the url to "&scope=chat_login+channel_subscriptions" because getting subscription info was my main goal at the time. I pressed enter and it gave me a new OAuth. I checked the new OAuth with the twitchAPI to see if it had indeed the scope I needed.


If you look at line 14, as you can see the needed scope is indeed there. Mission accomplished lol. The next thing I wanted to do was make it easy for people to create their own custom OAuth key using the existing key generator. I do not know of an existing easy to use method of making custom OAuth keys and so made my own here.

Finally here is the code snippet of the OAuth customizer used here (HTML + JavaScript):



Friday, March 14, 2014

XwitchBot

   I was asked to make a bot for twitch and so I did. It is called Xwitchbot (pronounced switch bot). I wouldn't call it a very good bot personally but it works. It provides some message handling, and is plugin based. Included is a single plugin that I made called Raffle2. As the name implies its used to hold raffles on twitch chats. To make plugins for this bot simply implement the interface "Globals.dll".

XwitchBot Files: https://www.mediafire.com/folder/aflc46sawa3rg/Xwitch
XwitchBot Download: http://www.mediafire.com/download/5oqmvpv3q51so78/XwitchBot.zip

Plugin Interface: http://www.mediafire.com/download/c80t7k4neh799cj/Globals.dll

Keep in mind that this is the first ever plugin based application I have ever made so it might lack some common practices etc. I am also mostly self taught and the implementation might not be optimal. All in all it works and I just wanted to share.

Doc:
The system is as follows. The main application loads the plugins in a pool of shared resources. That pool holds the IrcClient, message handler, and plugins as well as plugin resources. The IrcClient, MainUI, and PluginManager/Messagehandler are all on separate threads. Plugins can be on their own separate threads too if they are made so.

Here are some signatures and their description of the functions available in Globals.dll to make creating plugins easier.

Public Shared Function GetToSend() As String
Returns the first element in the list of messages to be sent to the server, and removes it from the list.

Public Shared Function GetRecvMsg(Optional ByVal RemoveOnGet As Boolean = True) As String
Returns the first element in the list of messages received by the server,
if RemoveOnGet is true remove the message from the list.

Public Shared Function GetConsoleMsg(Optional ByVal RemoveOnGet As Boolean = True) As String
Returns the first element in the list of messages to be shown as output,
If RemoveOnGet is true remove the message from list.

Public Shared Function ParseMessage(ByRef sourceInfo As Message, ByVal Input As String) As String
Returns a string replacing all fields with their respective information. sourceInfo is the Message where this function will get the required information from.

Public Shared Sub AddResource(ByVal newResource As Resource)
Adds a new Resource to the pool of resources.

Public Shared Sub AddResource(ByVal newResourceOrigin As String)
Adds a new Resource to the pool using a name.

Public Shared Sub RemoveResource(ByVal Origin As String)
Removes a Resource from the pool with the specified name (Origin)

Public Shared Sub RemoveResourceAt(ByVal index As Integer)
Removes a Resource from the pool at the specified index location

Public Shared Sub AddResourceElement(ByVal ResourceIndex As Integer, ByVal Type As String, ByVal Name As String, ByVal Value As Object)
Adds a resource element  to a resource in the pool at a given index location. Type, Name, and Value as specified.

Public Shared Sub AddResourceElement(ByVal ResourceOrigin As String, ByVal Type As String, ByVal Name As String, ByVal Value As Object)
Similar to the other but instead adds a resource element to a resource of a specified name (Origin)

Public Shared Sub RemoveResourceElementAt(ByVal ResourceIndex As Integer, ByVal ElementIndex As Integer)
Removes a resource element givent a resource index and the element index.

Public Shared Sub RemoveResourceElement(ByVal ResourceOrigin As String, ByVal Name As String)
Similar but removes a resource element with a specified name at the resource with the specified Origin

Public Shared Sub SetResourceElement(ByVal ResourceIndex As Integer, ByVal ElementIndex As Integer, ByVal Value As Object)
Given a resource index and a resource element index, change the value of the element.

Public Shared Function GetResourceElement(ByVal ResourceIndex As Integer, ByVal ElementIndex As Integer)
Returns the value at a specified resource element index at a resource index

Public Shared Sub AddToSend(ByVal msg As String)
Adds a message to the list of message to send to the server

Public Shared Sub AddToRecv(ByVal msg As String)
Adds a message to the list of messages that are received by the server

Public Shared Sub AddToConsole(ByVal msg As String)
Adds a message to the list of message that will be shown on the output

Public Shared Sub SetCurrentChan(ByVal newChan As String)
Sets the current channel the bot is in "Does NOT actually change the channel is in, used for message handling"

Public Shared Function GetCurrentChan() As String
Gets the current channel the bot is currently in "Does NOT actually get the channel the bot is currently in, used for message handling"

Public Shared Function GetResourceIndexOfOrigin(ByVal Origin As String) As Integer
Returns the index of a resource of a given name (Origin)

Public Shared Function GetElementIndexOfName(ByVal Name As String) As Integer()
Returns the resource index and element index for an element with a specific name

Public Shared Function GetElementIndexOfName(ByVal ResourceIndex As Integer, ByVal Name As String) As Integer
Returns the index of a resource element in the resource at the specified index

Public Shared Sub StopPluginManager()
Stops the plugin manager built into the bot and is used by default.

Public Shared Sub StartPluginManager(ByVal Owner As String, ByVal Server As String)
Starts the plugin manager built into the bot and is used by default.

Public Shared Sub StopConnectionClient()
Stops the IrcClients connection to the server

Public Shared Function StartConnectionClient(ByVal Server As String, ByVal Port As String, ByVal Nick As String, ByVal Channel As String, ByVal Pass As String, ByVal JoinMsg As String) As Boolean
Starts the IrcClients connection to a server, the return boolean indicates a successful or failed connection attempt.

The plugin must implement 3 things:
    ReadOnly Property Name() As String
    Function ProcessMessage(ByVal InputMessage As GlobalResources.Message) As Boolean
    Sub Config()

Messages are sorted into 2 types:
1) PRIVMSG (private message that is properly formatted in a way this bot can understand by default)
2) NS (non-sense/not-supported)

NS messages are not discarded, they are passed to all plugins (unless a plugin stops the chain) just like every other message, the difference is their type will be NS and a plugin would have to manually process that message in its raw form.

Messages that have been processed have a structure of the following:
    Public Structure Message
        Dim Type As MessageType
        Dim Nick As String
        Dim User As String
        Dim Host As String
        Dim Target As String
        Dim Message As String
        Dim Owner As String
        Dim Server As String
        Dim Raw As String
    End Structure

Message is where the message itself (what the user specifically send) is stored.
Raw is the message exactly as it was received by the bot with all heading information intact.

ResourceElements have the following structure
    Public Structure ResourceElement
        Dim Type As String
        Dim Name As String
        Dim Value As Object
    End Structure

Resource has the following structure
    Public Structure Resource
        Dim Origin As String
        Dim Resources As List(Of ResourceElement)
    End Structure

Here is the starting source for making a plugin:

For a complete source of Globals.dll :

Here is the source to my Raffle2 Plugin:

Source for UserList plugin:

And finally my horrible source for the bot itself:

As always if anyone has any issues or improvements please leave a comment :)

Wednesday, February 26, 2014

Saving bandwidth while chatting using UTF8 (Fixed)

UPDATE: Decode function fixed and tested (issue was a 4 was used instead of 1, silly me lol). Next up is maybe porting to C#,C++ and maybe Java, but its pretty simple I think people can do it themselves.

UPDATE: Well this is embarrassing, my new and faster decoding function is flawed lol, for now use the old decoding function for accuracy while I look into fixing my faster code.

UPDATE: New Encoding/Decoding Function and changes are below. From testing Encoding is over 5X faster than the previous code and Decoding is over 15X faster. Major change would be I implemented a lookup table so not so much string manipulation is required. Minor changes are I swapped and reordered some things around in favor of faster alternatives like for ex using index of instead of contains.

Test String (100,000 loop) = This is a test to try to make I can make my encoding and decoding for my scheme faster, it badly needs a speed boost lol!!!

OLD:
Encoding: ~9,334 ms
Decoding: ~13,556 ms

NEW
Encoding: ~1,732 ms
Decoding: ~849 ms


    Previously I had a concept of a modified UTF8 encoding scheme with selected characters being optimized. I had code but the scheme was broken to some extent and code flawed. I left it alone for a couple months and then pbanj(from irc) posted a message in all hex lol. At first I didn't know what to do with it, so I told him "wanna decode that for me?" and he said why would he have to. At that moment it hit me, he was using an app I made awhile ago with the encoding scheme I made (still with bugs). I decoded the message and the message said "has this been updated" and that was when I got back to work on fixing this scheme. The concept was simple, most English messages are simply of Latin characters as in a-z and A-Z. I thought all that was needed to represent these were 6 bits. But with 6 bits I had extra room for more characters so I squeezed in the next possibly most used characters, digits 0-9. I then had room for 2 more spaces for characters, I used one to represent a space because come on... its used to separate words from one another. The last bit is reserved for something that isn't a character. It is used instead as an escape or wildcard. The last spot in the 6 bit encoding (111111) is used to represent a non optimized character (UTF8). Because most English messages will contain a ratio of greater Latin letters and numbers 0-9 plus some spaces compared to punctuation and other special characters this scheme will most likely save bandwidth. This scheme works because UTF8 has certain characteristics and rules that compliment the workings of this scheme. A character encoded in UTF8 cannot (in bits) start with 10XXXXXX. The first byte contains information on how many bytes ahead needs to be read for the current character, for example 111XXXXX has three 1 bits leading so the character has 3 bytes total to represent that character. So including itself it needs to also get the next 2 bytes to successfully decode the character.

    Now Lets see how we can optimize UTF8 a bit. Lets assume we want to encode only with 6 bits, those 6 bits representing the optimized characters mentioned above. And every time a special character needs to be encoded the bits (111111) is used. That Character is then encoded using UTF8 and its byte representation is put aside later for use (we will get back to it later). The encoding then continues onto the next character, again if the character is not an optimized character then the bits (111111) is added and its UTF8 bytes are added to the end of the UTF8 bytes from earlier.

    At the end of the encoding we put everything together. We should have 2 groups of data now, the bytes of the characters encoded in UTF8 and the bits for the characters encoded with 6 bits. We need to put them together but how will the decoder know where the UTF8 bytes end and where the 6 bit encoding start. Earlier we established that UTF8 cannot start with 10XXXXXX, well we will use this to signify the end of the UTF8 but instead of the whole byte we only use the first 2 bits (1 and 0) because there is no point in reading the whole byte if UTF8 wont be able to decode it anyways. So 1 and 0 are placed in between the bytes and the 6 bit encoding. The last thing to do to encoding the message is convert it to bytes. We do this because the final data needs to be in a byte array, we do this by simply adding 1s until the number of bits in the final data is a multiple of 8 (8 bits are in a byte). 1s are used so the decoder doesn't mistakenly decoded any other character if the padding was 6 bits long, for example if 0s were used for padding and the padding was 6 bits long the first optimized character would be decoded at the end when the original message didn't have it.

    To decode is fairly simple. We read the message byte per byte first for UTF8 decoding and only when a UTF8 starts with 10 does it stop, the UTF8 characters are put aside for later use. The bits per bit decoding now starts at the beginning of the byte which started with 10XXXXXX. First thing to do is because the 1 and 0 were just used as markers they are skipped or removed.The rest of the message is decoded 6 bits at a time until there are no more bits to read or there are less than 6 bits remaining. Each 6 bits corresponds with one of the optimized characters. Each time the 111111 comes up it grabs the corresponding index of the special characters. For example if this is the first occurance of 111111 then that character in the encoding is the first character in the special characters we decoded earlier.

Here is a demo of the encoding:
Optimized characters: "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 " no quotes.
Message = Hello my name is TizzyT.
Hex = 2E:A1:10:B2:CE:F8:C6:3E:34:03:04:F8:84:BE:B4:86:59:62:DF:FF
Octets =
00101110 10100001 00010000 10110010
11001110 11111000 11000110 00111110
00110100 00000011 00000100 11111000
10000100 10111110 10110100 10000110
01011001 01100010 11011111 11111111

First we look at the first byte (first octet): 00101110
We see that it starts with a 0 meaning that this character only needs the byte to decode the character. This character in turns decodes to be the period at the end of the original message. We add that to a list of special characters for later use.

We move onto the next byte (second octet): 10100001
It starts with a 1 and 0 so from this we know to stop decoding in UTF8.
Note if it doesn't start with 10  continue decoding in UTF8.

We skip (or remove) the 1 and 0 because they are markers and look at the next 6 bits: 100001
That is the number 33, so we know the character here is the 33rd character in  our optimize characters list (count started from 0) which is "H". And we repeat this throughout the rest of the message.
000100 = "e"
001011 = "l"
001011 = "l"
001110 = "o"
111110 = " "
001100 = "m"
011000 = "y"
111110 = " "
001101 = "n"
000000 = "a"
001100 = "m"
000100 = "e"
111110 = " "
001000 = "i"
010010 = "s"
111110 = " "
101101 = "T"
001000 = "i"
011001 = "z"
011001 = "z"
011000 = "y"
101101 = "T"
111111 = Special Character
111111 = Special Character

Here we come across 111111 which is to signify that the current character is a special character and it is the first occurrence so we look to the first decoded special character we have (which is the period).
Now we still have 6 more bits to read so we read those, if there were less then 6 bits left you stop because it is the end of the encoding and the remaining bits are there as padding. If the last six bits are 111111 the encoding ends because in such a case the bits are used as padding otherwise decode as usual.

Using this scheme we effectively saved 4 bytes with this message.
            UTF8 = 48:65:6C:6C:6F:20:6D:79:20:6E:61:6D:65:20:69:73:20:54:69:7A:7A:79:54:2E
This Scheme = 2E:A1:10:B2:CE:F8:C6:3E:34:03:04:F8:84:BE:B4:86:59:62:DF:FF

In the best case scenario (using only optimized characters and no padding) messages are 0.75 the size of  the same message encoded in standard UTF8. In the worst case this scheme can be any number of bytes larger, in which case the message should be sent using the standard UTF8 encoding. Because this scheme is based off UTF8 and the fact that this scheme decodes using UTF8 first there is no conflict with using this scheme to decode a standard UTF8 message.

Download Demo App: http://www.mediafire.com/download/x0l1cyf1zvxgy9x/Encode_Decode.exe
Note: The Demo App doesn't fall back to standard UTF8 encoding to show worst case scenario of a given message.

After thoughts:
1) Why didn't I use compression instead of this encoding scheme?
    I have thought of that and did several test and concluded that while compression with large messages with hundreds or thousands of characters would work nicely, in many or most cases much nicer than this scheme. In an instant messaging environment people don't hold their messages until there is a huge bulk..I mean come on its called instant messaging for a reason, people send whats on their minds as soon as they finish typing it and for such short messages most if not all compression actually makes the message larger in size.

2) Why did you make this?
    I was first intending to make a new chat program just for the heck of it and while doing so I needed to decided what encoding scheme I would use to send text messages. I had 2 factors that I wanted to keep optimal. The first was to be able to use a large range of characters and the second being it had to use as little bandwidth as possible. The first obvious choice was to use ASCII but that later turned out to be very limited in what I wanted users to be able to type. The next was UTF8. While looking into how UTF8 worked and how it came to be I was amazed as to how flexible it was. It only uses the required amount of bytes needed to represent a given character and thus my knowledge on variable byte encoding grew and of course the scheme I came up with is essentially taking that concept a bit further. So back to the original question, this scheme satisfies both factors and saves enough bandwidth in my opinion for me to opt for its use, and in the worst case scenario in a completed application (if made to) the message would be the size of a standard UTF8 encoded message.

3) Have you tried to optimize it even more?
    I have thought about it but besides rearranging some declarations around etc I haven't really changed the fundamentals of my code. There are probably optimizations here and there but I haven't gotten into doing that, like maybe making use of bit shift operators or creating a look up table instead of constantly calculating things. Those things I will maybe get around to later for now I am satisfied with the basic concept and workings of the scheme and I'll leave the optimizing and/or porting for others who want to use it.

4) You say this saves bandwidth but how when some messages aren't smaller in size?
    When I say save bandwidth I am talking about traffic used within a given duration. For many people that duration is at the start of a billing cycle to the end of one. Here in America and probably most other countries a billing cycle is usually on a per month basis. So within a month this scheme would save you bandwidth. For a single message sure it might not be that much saved but over hundreds or even thousands of messages throughout the month you would have saved a noticeable amount in my opinion.

5) Why 6 bits? Why not lower?
    I will be honest I wanted to use 5 bits at first and being a complete scatter brain I completely forgot about capital letters so it took me a good half hour or so before my brain fart subsided and increased it by 1 bit. There are 26 letters in the Latin alphabet but there are also capitalization so already there are 52 characters that need to be represented and that exceeds the 32 possible with 5 bits. And I would argue that with 6 bits it works out nicely with the whole alphabet with capitalization, digits 0-9, a space and the required escape/wildcard.

Here is the updated code I have for this scheme which encodes 5X faster than the old one and decodes about 15X faster than the old one.
(100,000 loop stopwatch as benchmark with string of 123 characters):


NOTE:
While this scheme can save bandwidth and is perceived fast, it is much slower than standard UTF8 encoding and requires more steps. The encoding for practical uses though should be more than fast enough.

As always if anyone has any issues or improvements please comment.
Open to non commercial use.

Saturday, February 15, 2014

Large Prime Number Generator

   I was using a combination of code from my previous posts etc and while looking at the code, I realized something, why do I need to repeat so much of it? I thought I could possibly optimize this a bit (Method still slow as crap). The first thing I did was make it so that each new random number generated will all have the same length in bits. Because they are all the same length in bits I don't have to generate new random numbers each time for the potential witnesses. Another thing was I can just set the length of bits instead of using the not so accurate tobytearray.length or using my own function to get bit length, I can just set it. I also added a multiple of 5 check before the iterations so that it doesn't try checking a number that ends in 5 (because primes larger than 2 only end in 1,3,7,9) instead it makes the number end in 3 by subtracting 2. For ease iterations can be set manually, if not the number of iterations is the square root of the length.

Code Note: Line 27 can be changed to:
"PrimeGen -= X" *X can be any even number
Depends on what you want the function to do when the current number isn't a prime.

Note: As always any errors or improvements please post in comments.

Miller Rabin VB.net

   Here is an implementation of Miller Rabin Primality Test. It was originally in C# and was converted to VB.net with some modifications and additions.

Note: If there is anything wrong or can be improved please comment. I am also not 100 percent sure this is Miller Rabin it is what I found as I was searching for Miller Rabin, but this is still a Primality Test (more accurately this is a composites test).

BigInteger Bit Length

   I was looking for Miller Rabin implementation in .Net (vb or c#) and found one. I went through the code and apparently I needed to generate a random BigInteger hence why I made THIS. Another thing I noticed was the algorithm needed that random number to be the same length as the number to be checked. Now I don't know how precise this length had to be whether it was just bytes or bits. But I saw something I didn't like and I sought out to 'change' it (might not be broken to say 'fix').

Number = 8739
In Hex = 2223
Actual = 15
Report = 16 (using BigInteger.ToByteArray.Length * 8)

That lack of precision kinda bugged me lol and so:
Note: If there is a better way (which I'm sure there is) please let me know in the comments.

Random BigInteger Generator

While I was working with primes I needed to generate large pseudo random numbers. I looked and I was envious that Java had one and from what I saw .Net didn't have one built yet? I decided to make my own and to share.

Random BigInteger Generator (VB.net):

PS: If there is an error please let me know also if there is a faster or better way please comment :)

Prime Sieve

Found a Prime Sieve function online and tested it, took ~3570ms to generate primes up to 10,000,000. Was pretty fast compared to the one I made awhile back so I decided to make a new one tailored to be faster than the one I just found. My final version takes ~2790ms to generate primes up to 10,000,000. Code I found was originally in C# but I converted it to VB.net.
Note: Speeds may vary per machine.

Sieve I made with use of multiplication instead of modulus (even faster than stated above):

Sieve I made:

Sieve I found:
Note: Any errors or improvements please post in comments.

Friday, January 31, 2014

MangaHere Downloader

[UPDATE JAN 24 2016] This no longer works. MangaHere has implemented some delayed fetching method which loads the actual image after what seems to be a place holder image (its some GIF). This cant be simply fixed but because I have abandoned this app already I will not be providing any revisions, versions of this app.

Here is a tool which is a standalone of what my final program is going to be. This downloads mangas (completely or upto latest) from MangaHere.com. All downloaded Mangas are organized in their own folder and in their own chapters in the same location as this program. I think it is faster than MangaRipper but not sure.

Download: http://www.mediafire.com/download/9fh9bc83dxqr75i/MangaHereDL.exe
Usage: Run the program, type or paste the main url of a manga from MangaHere.com and let it do its thing.

Fixed:
1) Invalid Characters for file/folder names which caused program to throw an exception.
2) Added Capitalization algorithm to correct the all caps anime names from previous version.
3) Tried multiple ways of multi-threading and found one that seems to work best.
4) Added folder name ordering for mangas that have duplicate chapter names.
5) Fixed an issue where program completes before all scans are downloaded.
6) Fixed issue where if the number of chapters is less than number of threads, the other threads are
    left idling.
7) Some other things I can't remember at the moment.....

Preview:






















Monday, January 13, 2014

MangaHere Search Tool

   This isn't really the initial goal of the program I was working on but I decided that I should make a separate standalone tool for searching MangeHere. This was originally supposed to be a plugin for my manga reader I'm working on but I decided to put out a smaller app just for the heck of it.

Download : http://www.mediafire.com/download/wvqgegqmcxqabjh/TzBrowse_MangaHere.exe

Usage: Run the program, enter what you are looking for and it will do its thing.
Double click to open the site associated with the manga. Press Esc to search for a new manga.

Info: Compatible with MangaHere, displays name and link.
Improvements: I plan to implement a multi-threaded and more optimized version in the final reader for performance as this standalone one is slow for searches that contain a lot of results, will also include more info like rating and summary etc.

 Image: http://i.imgur.com/EDEVzkL.png
Issues:
1) Single Threaded, kind of slow if search contains many results.