• Login
  • Register
  • Dolphin Forums
  • Home
  • FAQ
  • Download
  • Wiki
  • Code


Dolphin, the GameCube and Wii emulator - Forums › Dolphin Emulator Discussion and Support › General Discussion v
« Previous 1 ... 56 57 58 59 60 ... 368 Next »

Feature Idea: Augmented pointing with Wii Remote Plus
View New Posts | View Today's Posts

Thread Rating:
  • 1 Vote(s) - 3 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Thread Modes
Feature Idea: Augmented pointing with Wii Remote Plus
03-27-2018, 12:03 PM (This post was last modified: 03-27-2018, 03:54 PM by myownfriend. Edit Reason: Making it less like a wall of text )
#1
myownfriend Offline
Junior Member
**
Posts: 7
Threads: 1
Joined: Mar 2018
I've never looked at the raw data that comes out of the Wii Remote so forgive me for any gaps in knowledge about the subject and I'm sorry if there's someplace else I should have posted this.


However, I DO know that it can determine x, y, z, and roll with just the IR camera and sensor bar. For a regular Wii Remote, that's really all it has to determine those things but a Wii Remote Plus could (and does in some games) it's IMU (gyroscope and accelerometer) to do it's own less-accurate pointing without the sensor bar, so why not do that to fill in gaps in IR camera data?

In short

I'm suggesting that Dolphin provide the option to do IMU pointing while using the sensor bar, screen size, and sensor bar width data as a kind of sync/reference.

Why?

There's about 3 years of games that came out for the Wii before the Wii Remote Plus and Wii MotionPlus were released and, even afterwards, most games still didn't support it. This could enhance those games in a similar way to how a Wii Remote Plus might have.

What exactly would it fix?

I'm assuming sometimes when the Wii Remote is acting kind of erratic it's because the sensor bar is only partially out of view. Sometimes the Wii Remote might only see one tracking point in its view which isn't enough to determine depth and roll. With one sensor point still in view, it can still be used for pointing though and previous known depth and roll could work in conjunction with IMU data to determined/interpolate guess depth and roll in the current frame.  When both sensor points are out of view, that's when full IMU pointing would be in use.

Of course, it could also be used as backup for people who have a Wii Remote Plus and no sensor bar at all as long as they have a way of re-centering it.

Why should we care about screen size and sensor bar width?

If Dolphin knows the screen size and sensor bar length then it could theoretically be used along with the last known depth data to determine how sensitive gyro aiming should be. For example, if a player is closer to the sensor bar and they're playing on a larger screen then they will tend to want to tilt the Wii Remote more to point to the edges of the screen. In this case, IMU pointing would be made less sensitive. When they're further away they will tilt the controller less so gyro-pointing sensitivity can be increased.

Now if someone is using a tablet that has a screen the same width as the sensor bar, then they would never tilt the controller all much. That would also be a scenario where someone might prefer to make their own, smaller sensor bar solution since it would more likely that one or no point on a standard sensor bar is in view. If this works correctly, the player wouldn't notice when they've been switched to IMU pointing as long as the controller sees the sensor bar soon afterwards.

Issues

From what I've seen, using accelerometers to try to determine depth even if based off a previous reference would very quickly lead to errors stacking up, so it would probably be best to only try doing that for a few frames and might not be worth doing at all.
Find
Reply
03-27-2018, 04:42 PM
#2
JMC47 Offline
Content Producer
*******
Content Creators (Moderators)
Posts: 6,542
Threads: 29
Joined: Feb 2013
When we're interfacing a Wii Remote with the Wii, specifically in our passthrough modes, we don't have any access to talk with the Wii Remote. The game is talking directly.

While what you're saying is probably possible, it'd require a pretty large rewrite of an already messy part of the codebase.
Find
Reply
03-27-2018, 05:25 PM (This post was last modified: 03-28-2018, 02:02 AM by mbc07.)
#3
mbc07 Offline
Wiki Caretaker
*******
Content Creators (Moderators)
Posts: 3,563
Threads: 47
Joined: Dec 2010
Also, every game can potentially "talk" to the Wiimote differently, so it's probably impossible to implement this in a way that just works with all games that don't require the Motion Plus. Then you would need a lot of game-specific modification/hacks and it would become a mess...
Avell A70 MOB: Core i7-11800H, GeForce RTX 3060, 16 GB DDR4-3200, Windows 11 (Insider Preview)
ASRock Z97M OC Formula: Pentium G3258, GeForce GT 440, 16 GB DDR3-1600, Windows 10 (22H2)
Find
Reply
03-27-2018, 05:25 PM
#4
mstreurman Offline
Above and Beyond
*******
Posts: 1,239
Threads: 11
Joined: Nov 2015
(03-27-2018, 12:03 PM)myownfriend Wrote: I've never looked at the raw data that comes out of the Wii Remote so forgive me for any gaps in knowledge about the subject and I'm sorry if there's someplace else I should have posted this.


However, I DO know that it can determine x, y, z, and roll with just the IR camera and sensor bar. For a regular Wii Remote, that's really all it has to determine those things but a Wii Remote Plus could (and does in some games) it's IMU (gyroscope and accelerometer) to do it's own less-accurate pointing without the sensor bar, so why not do that to fill in gaps in IR camera data?

In short

I'm suggesting that Dolphin provide the option to do IMU pointing while using the sensor bar, screen size, and sensor bar width data as a kind of sync/reference.

Why?

There's about 3 years of games that came out for the Wii before the Wii Remote Plus and Wii MotionPlus were released and, even afterwards, most games still didn't support it. This could enhance those games in a similar way to how a Wii Remote Plus might have.

What exactly would it fix?

I'm assuming sometimes when the Wii Remote is acting kind of erratic it's because the sensor bar is only partially out of view. Sometimes the Wii Remote might only see one tracking point in its view which isn't enough to determine depth and roll. With one sensor point still in view, it can still be used for pointing though and previous known depth and roll could work in conjunction with IMU data to determined/interpolate guess depth and roll in the current frame.  When both sensor points are out of view, that's when full IMU pointing would be in use.

Of course, it could also be used as backup for people who have a Wii Remote Plus and no sensor bar at all as long as they have a way of re-centering it.

Why should we care about screen size and sensor bar width?

If Dolphin knows the screen size and sensor bar length then it could theoretically be used along with the last known depth data to determine how sensitive gyro aiming should be. For example, if a player is closer to the sensor bar and they're playing on a larger screen then they will tend to want to tilt the Wii Remote more to point to the edges of the screen. In this case, IMU pointing would be made less sensitive. When they're further away they will tilt the controller less so gyro-pointing sensitivity can be increased.

Now if someone is using a tablet that has a screen the same width as the sensor bar, then they would never tilt the controller all much. That would also be a scenario where someone might prefer to make their own, smaller sensor bar solution since it would more likely that one or no point on a standard sensor bar is in view. If this works correctly, the player wouldn't notice when they've been switched to IMU pointing as long as the controller sees the sensor bar soon afterwards.

Issues

From what I've seen, using accelerometers to try to determine depth even if based off a previous reference would very quickly lead to errors stacking up, so it would probably be best to only try doing that for a few frames and might not be worth doing at all.

I'm no developer either, but I don't think this will ever happen because:

1. Emulated Wiimote mode: To connect it to Windows you need a driver, it will connect using Bluetooth but your computer won't know what to do with it. There are drivers that allow you to use the Wiimote as a mousepointer, but AFAIK none of them expose the Gyro data to Windows.

2. Real Wiimote mode: Would be the only way I think it would be possible to implement it, because it emulates the Bluetooth adapter from the Wii, so somehow you might be able to hook into that and add a few extra steps and convert Gyrodata to IR data. You have to remember that the games themselves do the interpretation of the data that comes in and the games don't know what a Motion plus is and thus do not know what data that Gyrodata is used for...

3. Bluetooth passthrough: "Nothing" is emulated in the controller path, it literally gives full control of your computers Bluetooth module to the Emulated Wii/Game so it works exactly the way it would work in a real Wii. I don't think you can hook into that and add extra information to the stream.

4. New "motionplus enhanced" mode? I think, considering the previous points that this also wouldn't be a viable option.
Check my profile for up to date specs.
Find
Reply
03-27-2018, 09:07 PM
#5
myownfriend Offline
Junior Member
**
Posts: 7
Threads: 1
Joined: Mar 2018
I'm sure there will be some games that might not be completely compatible with this but as an optional setting that doesn't seem like a big deal to me. I'm also very sure it would be difficult to implement but I find it hard to believe that it would be a mess (in regards to per-game hacks etc) or impossible to implement at all. With Emulated Wii Remote there's already options to use an analog stick to update the IR camera's position incrementally. The augmented pointing thing would be more complex than that but still similar in the respect that it's preprocessing data before it's sent to the game and really shouldn't be any less compatible.

As for the games processing the data themselves, I don't see why that would be a huge issue. The modified IR data wouldn't be giving the game any data that a real Wii wouldn't be able to give it nor would it be in any strange structure. If the pointer goes off-screen it would still give it the same data that a normal Wii Remote would. The only difference is that if a point's X,Y position data disappears due to obstruction when the points is only half-way down the screen, it would be able interpolate the missing position data using the IMU until the point(s) is visible again.
Find
Reply
03-28-2018, 01:07 AM (This post was last modified: 03-28-2018, 01:16 AM by mstreurman.)
#6
mstreurman Offline
Above and Beyond
*******
Posts: 1,239
Threads: 11
Joined: Nov 2015
(03-27-2018, 09:07 PM)myownfriend Wrote: I'm sure there will be some games that might not be completely compatible with this but as an optional setting that doesn't seem like a big deal to me. I'm also very sure it would be difficult to implement but I find it hard to believe that it would be a mess (in regards to per-game hacks etc) or impossible to implement at all. With Emulated Wii Remote there's already options to use an analog stick to update the IR camera's position incrementally. The augmented pointing thing would be more complex than that but still similar in the respect that it's preprocessing data before it's sent to the game and really shouldn't be any less compatible.

As for the games processing the data themselves, I don't see why that would be a huge issue. The modified IR data wouldn't be giving the game any data that a real Wii wouldn't be able to give it nor would it be in any strange structure. If the pointer goes off-screen it would still give it the same data that a normal Wii Remote would. The only difference is that if a point's X,Y position data disappears due to obstruction when the points is only half-way down the screen, it would be able interpolate the missing position data using the IMU until the point(s) is visible again.

1. Are you (or a developer you know) going to write a driver for Windows (and possibly other OSes) that can interface with the Gyro in Motion plus and convert this data to an IR stream and then combine it with the existing IR data? If your answer is No then Emulated Wiimote is out of the question.
2. The only way that I see this as remotely viable would be with the Real Wiimote option since we Emulate the Bluetooth there and thus would be able to tell the Bluetooth to also accept Gyro data as IR data and somehow convert Gyro to IR movement and combine both into 1 stream.
3. Passthrough will forever be out of the question because we do not have any control over the Bluetooth or the Wiimote

And yes, adding Gamespecific hacks will make the codebase very messy, less readable and the devs here (nor do I) have any interest in glitchy game specific hacks, also to enable these game specific hacks it needs to be done manually (what if I want it on for most games except for 1 or 2 because I do not like what it does) and would need space in the User Interface for every game which would have to go through @MayImilae.


-OFFTOPIC-

I love it how non-developers think these kind of things are so easy to do... It is like a Service Desk Support Agent talking to a Customer (real life example, sadly enough Sad )

- Customer: I need an account to log in to Windows.

- Agent: Okay, please file this request and we will get to it.

- Customer calls back 5 minutes later: Why doesn't my account work yet?

- Agent: You just filed the request, it takes some time for everything to be created

- Customer: That is bullshit, you guys are just lazy, you just have to push the button for it to work, you are all against us.

- Agent: Let me explain: It takes some time for it to be created and then some time for it to synchronize, then we have to create a Homedrive on the server, allocate space to it and link it to your account, on top of that you also need an e-mail address so that also has to be created on the servers, have space allocated to it, link it to your account and then inform the DNS server that this is now a valid address. Then the roaming profile must be created and all applications that you need must be attached to it. This not only takes time but also people and system resources to do this. If we would do this during day times it means it will slow down the network for everyone that is currently using it. That is why synchronizations are done in batches at night when no one is using the network.

- Customer: *mutters under his breath* Lazy Assholes and slams down the phone angrily.



User calls back after account is created

- Customer: The account doesn't work

- Agent: *checks account and sees nothing wrong and is even able to log in to it* What doesn't work on the account?

- Customer: The account I got doesn't work on my Laptop when I am at home.

- Agent: Ok, are you using a company laptop or a private laptop

- Customer: It is my private laptop

- Agent: What you want is not possible at our company

- Customer: But my colleague is able to access everything from home on his laptop

- Agent: Then he probably has a company laptop

- Customer: Yes he does, but I don't want one of those old and slow laptops I bought this laptop specifically because it is fast and I need it to work from home. You just have to push the button to make it work, it is so easy for you all to say it is not possible. Just do it.

- Agent: Sir, this is not possible at our company. You need one of our laptops and then you will be able to access the network and your account from home.

- Customer: But a friend who works at Company X was able to log in from home with his own private computer.

- Agent: Our network isn't the same as the network at Company X, if we would start to allow everyone to bring their own personal device the network and servers would become convoluted and a mess and in the end we wouldn't know anymore what computer is still in use, if it is a private or a company laptop and so on.

- Customer: *Yelling angrily*IT WOULD NOT MAKE A MESS IT WOULD MAKE THINGS EASIER, BUT IT IS ON THE INTERNET, IT IS THE SAME EVERYWHERE SO IT SHOULD BE EASY TO MAKE IT WORK...

- Agent: .... Sorry sir, what you want to do is simply not possible. Is there anything else I might be able to help you with?

- Customer: *Slams phone down*
Check my profile for up to date specs.
Find
Reply
03-28-2018, 02:09 AM
#7
mbc07 Offline
Wiki Caretaker
*******
Content Creators (Moderators)
Posts: 3,563
Threads: 47
Joined: Dec 2010
(03-27-2018, 09:07 PM)myownfriend Wrote: [...]

As for the games processing the data themselves, I don't see why that would be a huge issue. The modified IR data wouldn't be giving the game any data that a real Wii wouldn't be able to give it nor would it be in any strange structure. If the pointer goes off-screen it would still give it the same data that a normal Wii Remote would.

The Wiimote has various reporting modes, and the data format returned is different between these modes. AFAIK, to get Motion Plus data, you need to switch to any of the interleaved modes, and that affects the quantity of data returned and also latency (as it needs two reports to send the same amount of data it was previously sending in one report plus the Motion Plus data, and some least significant bits of the accelerometer axes are also dropped on this mode). It's unpredictable how the games would behave and some would certainly break...
Avell A70 MOB: Core i7-11800H, GeForce RTX 3060, 16 GB DDR4-3200, Windows 11 (Insider Preview)
ASRock Z97M OC Formula: Pentium G3258, GeForce GT 440, 16 GB DDR3-1600, Windows 10 (22H2)
Find
Reply
03-28-2018, 07:43 AM (This post was last modified: 03-28-2018, 07:46 AM by JulianLoehr.)
#8
JulianLoehr Offline
Junior Member
**
Posts: 49
Threads: 1
Joined: Nov 2015
Also the IR-Data from the Wii Remote is sent as up to four IR-points detected by the camera. That means up to four X-Y coordinate pairs. Those coordinates have min-max values, that are exactly the size of the camera field of view. So if the sensor bar or one of its points is moved outside of the camera view, you have no range of values left to go beyond the camera view.

In addition to that, the Wii Remote does not interpret the IR-Data nor sends X,Y,Z values. Instead as written above IR-Point data is send and the interpretation is up to the game.
Creator of HID Wiimote: A Windows Device Driver for the Wii Remote
Website Find
Reply
« Next Oldest | Next Newest »


  • View a Printable Version
  • Subscribe to this thread
Forum Jump:


Users browsing this thread: 1 Guest(s)



Powered By MyBB | Theme by Fragma

Linear Mode
Threaded Mode